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Executive Summary 



Purpose of White Paper 

Under the provisions of the School-to-Work Opportunities Act of 1994 (STWOA), the state of 
California contracted with WestEd and MPR Associates to conduct a statewide evaluation of its 
School-to-Career (STC) system. Findings from the evaluation will be used to support further 
development of this system and serve as a key information source for the many federal, state, and 
local stakeholders who are working to improve our education system. The overarching research 
questions to be addressed by the statewide evaluation study are: 

• What is the status of STC implementation in California? 

• How has STC affected student preparation for post-secondary education and career entry? 

• To what degree and in what ways has STC contributed to systemic change? 

• Have STC principles penetrated the community deeply enough to be sustainable? 

The main goal of this White Paper is to lay the foundation for the evaluation to be conducted 
from Fall 2000 through June 2002. Using a variety of sources — including Local Partnership (LP) 
applications for STWOA funds, LPs' Quarterly Reports, data from two federally-sponsored 
longitudinal evaluations of STWOA implementation, and existing evaluations conducted for 
LPs — ^WestEd and MPR have examined the status of STC in California. This Paper discusses 
these findings and emphasizes how they relate to designing a feasible evaluation strategy. 



The Evolution of School-to-Career in California 

An historical overview of STC helps to frame current efforts in California as the state works to 
create and sustain its STC system. Since the landmark Smith-Hughes Act of 1917 that supported 
vocational education programs aimed at helping low-performing students prepare for 
occupations, efforts related to preparing students for the workforce have changed dramatically at 
the national and state levels. In the 1990s, the goals of vocational education programs began to 
change and a new STW model emerged. In contrast to federally funded vocational education 
programs supported by the Smith-Hughes Act, current STW programs are designed to develop 
academic and work-based skills for all students. While there are many different approaches to 
STW, all have at their core the goal of improving academic achievement while at the same time 
providing students with marketable, work-based skills. 
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Major Legislation and National Reform Initiatives Supporting STW 

Recent federal legislation has had a significant impact on California's STC implementation. For 
example, the Carl D. Perkins Vocational and Applied Technology Education Act of 1990 
(Perkins II) helped create the new vision that ultimately became the STW model by promoting 
"reform, innovation, and continuous improvement in vocational and technical education" 
(Summary of Public Law 105-332). One particularly noteworthy aspect of the Perkins II 
legislation is set-aside funding under Title III (E) for special Tech Prep grants for high schools to 
work with two-year colleges or apprenticeship programs to create a coherent sequence of study 
for students that combines academic and vocational courses. Perkins III, which was authorized 
in 1998, further extends the new vision by promoting the development of integrated, "one-stop" 
education and workforce development systems at the state and local levels and placing more 
emphasis on assessment and accountability for student achievement relative to rigorous academic 
and industry standards. 

The legislation with most direct influence over California's STC system is the STWOA, whose 
chief purpose is underwriting the initial costs of establishing statewide STW systems in all states. 
Although certain core elements are expected (e.g., integrating school-based and work-based 
learning; connecting activities; career majors; linkages between secondary and post-secondary 
education), states are allowed considerable leeway in designing their STW systems. The 
STWOA also specifies that state STW systems should be integrated with systems developed 
under Goals 2000: Educate America Act and the National Skill Standards Act of 1994 in order to 
serve as a comprehensive educational reform strategy. 

The U.S. Department of Education also initiated related programs aimed specifically at high 
schools, such as the Comprehensive School Reform Demonstration and the New American High 
Schools. Like Perkins III and STWOA, these reforms include integration of academic and 
vocational curricula and the establishment of partnerships with employers. Moreover, influential 
high school reform networks, such as the Coalition of Essential Schools and the "High Schools 
that Work" project (established by the Southern Regional Education Board), support many of the 
principles underlying the STW movement. 

The School-to-Career Initiative in California 
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Various initiatives currently exist in California that were created to provide individuals with 
career awareness and employment-related skills. Some programs were specifically designed as 
opportunities for job training, such as those funded by the Job Training Partnership Act (JTPA), 
which provides services for economically disadvantaged adults and youth, dislocated workers, 
and others who face substantial employment barriers. Others have broader aims and emphasize 
the integration of school- and work-based learning, and promote curriculum integration, K-16 
articulation, and the involvement of employers, labor organizations, schools, parents, students, 
and other key stakeholders. Interestingly, many of these key programs, including California 
Partnership Academies, Tech Prep, and Regional Occupational Center and Programs (ROC/Ps), 
preceded California's successful application for STWOA funds in 1996. 

California began planning in 1993 for implementation of its STC system, and received its first 
STWOA implementation grant in 1996. Over a five-year period, California can receive up to 
$131.4 million to serve as venture capital to develop its STC system. 

The STC plan for California was created with the assistance of multiple stakeholders, including a 
27-member task force appointed by the governor and representatives from the California 
Department of Education, the Chancellor's Office of the California Community Colleges, and the 
Employment Development Department. The cornerstone of California’s STC plan is the funding 
and/or support of 59 LPs around the state. STWOA defines a LP as “a local entity that is 
responsible for local School-to-Work Opportunities programs” (Section 4(1) of STWOA). LPs 
are required to include employers; representatives of local educational agencies, post-secondary 
institutions, and organized labor; local educators; and students. To further support STC, 
California is undertaking other initiatives, including; development of an STC Curriculum 
Framework; establishment of a statewide assessment system to help ensure that students have the 
requisite academic foundation and incorporation of career-related assessments for high school 
students (Assessments for Career Education) into the statewide student assessment system; 
creation of the Golden Seal Merit Diploma, which will be awarded in part on the basis of 
excellence in core academic subjects and participation in work-based learning; and coordination 
with the state's One-Stop Career Centers. 
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In summary, California has received from the federal government and disseminated among LPs a 
substantial amount of seed money to implement a statewide STC system. However, much of the 
work related to career preparation in California predates the STWOA (California Partnership 
Academies, Tech Prep, ROC/Ps) or is funded by other concurrent funding streams (One-Stop 
Career Center System). One of California's major STC strategies, as stated in its STC 
implementation plan, is to effectively build on the existing efforts to create a comprehensive and 
efficient STC system. 

The National Context for Examining STC in California 

To assess California's STC effort, it is important to place the state's accomplishments in a 
national context. Understanding how the STW system has developed across the country since 
1994 will ultimately allow for a comparison between California's system and national results. 

Funded by the National STW Office, three major studies — ^the National Evaluation of STW 
Implementation, the STW Progress Measures, and STW in the 1990s; A Look at Programs and 
Practices in American High Schools — ^have collected and reported information on how students, 
schools, districts, employers, and communities are responding to the call for more 
comprehensive, coordinated school reform that links schooling to the world of work. 
Summarized below are the key findings from these three national studies with respect to key 
aspects of STW activities across the nation. 

Governance/Institutional Participants 

STW is an enterprise typically led by educational institutions in most states. Of the 37 funded 
states, 22 chose to locate the state STW office in the department of education. At the local level, 
school districts and high schools typically lead the STW effort, with the following levels of 
collaboration with other key groups: employers (most extensively), post- secondary institutions 
(less extensively), and organized labor (least extensively). 
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Career Development 

The national evaluations indicate that most states and LPs emphasize broad career development 
over other components of STW implementation (National Evaluation, p. xv). Other key 
observations about career development activities across the nation include the following: 

• In 1998, about four out of five seniors participated in interest inventories, employer talks 
at schools, and/or work-readiness classes, and two out of three participated in a job 
shadow or work site tour at some point during high school (Ibid., p. 62). 

• In 1997, about half of elementary schools in STW partnerships reported offering 
significant career information within the curriculum. One year later, this increased to 
about two-thirds of elementary schools (Progress Measures, p. 10). 

• In 1998, 68% of middle/junior high schools in reporting partnerships provided structured 
exploration activities, such as individualized learning plans linked to career pathways 
offered in high school. This compares with 59% in 1997 (Ibid., p. 1 1).' 



Promoting Student Access 

The National Evaluation found that STW partnerships have been relatively successful at 
promoting STW for all students, although participation rates among subgroups of students vary 
to some extent. Some of the findings of the National Evaluation's eight in-depth study states, 
include: 



• The activities promoted in the STWOA are engaging students at all academic 
performance levels. However, students in the bottom quartile of their class are slightly 
less likely than their peers to participate in career-related academics (9.5% versus about 
13% for the higher quartiles). 

• Students who completed a college preparatory curriculum are just as likely as students 
who had not completed such a curriculum to participate in comprehensive career 
development, workplace activities linked to school, and career-related academics. 

• Students who enrolled in a four-year college are slightly less likely than their peers who 
either did not enroll in college or enrolled in a two-year institution to participate in a 
workplace activity linked to school, but are very similar to their peers with respect to 
comprehensive career development and career-related academics. 



‘ Data Source: U.S. Department of Labor, Bureau of Labor Studies, National Longitudinal Survey of Youth, 1996- 
97, Survey of School Administrators; U.S. Department of Education, National Center for Education Statistics, 
Common Core of Data Survey, 1993-94. 
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Work-Based Learning 

Work-based learning takes many forms in U.S. high schools and funded STW partnerships, 
ranging from one-time visits to work sites — with very little connection to what students are 
learning in school — to paid youth apprenticeships that are the culmination of a coherent sequence 
of coursework. Not surprisingly, the more intensive types of work-based learning opportunities 
are the least common. While many high schools offer work-based learning opportunities, 
information from the Progress Measures Surveys suggest that only small percentages of students 
take advantage of these opportunities. Another important finding is that paid or unpaid positions 
obtained through school provide students with greater learning opportunities (National 
Evaluation, p. 89-90). Unfortunately, these opportunities are relatively rare. Only about one 
quarter of paid or unpaid positions that students held were obtained through school in 1998 
(National Evaluation, p. 86). 

Summary of National Context 

The three national studies of STW show considerable progress in only a few years toward 
implementing key elements of the STWOA. However, many schools are providing only some 
parts of the STW school-improvement strategy, and few students participate in a comprehensive 
career-focused high school learning experience. The National Evaluation indicates that schools 
have a long way to go to integrate comprehensive career development activities, career-related 
academics, and paid or unpaid work experience that is linked to learning in the classroom, with 
only about 3% of students having been involved in all three of these curriculum approaches. 
Moreover, although the goals of the STW movement are to support high academic standards and 
produce positive student outcomes, it is too early to identify the role that STW is playing in 
achieving these goals (National Evaluation, p. 139). In most states, implementing STW and 
raising standards and accountability are two important school improvement efforts that have been 
operating independently of each other. 

Efforts to Create a School-to-Career System in California 

Whereas the previous section focuses on the national context for STW, this section focuses 
attention squarely on California and the work of the LPs. Analyses of LPs' Performance 
Matrices, quarterly reports, and evaluation reports show that California gave LPs a considerable 
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amount of latitude in designing STC activities and strategies. Although it is difficult to draw 
inferences about activity levels within or across partnerships or discuss progress toward reaching 
specific goals based on these documents, these data provide a rich, qualitative description of LPs' 
efforts related to four areas of STC; school-based learning, connecting activities, work-based 
learning, and access for all students. 



School-Based Activities 

Data suggest that LPs are using a wide variety of school-based activities and strategies, 
including: 

• offering career exploration and college awareness, which typically begin in the 
elementary grades and continue through high school; 

• offering career counseling and guidance on career choice and opportunity rather than 
focusing almost exclusively on class scheduling; 

• raising student performance and academic standards by the elimination of "general 
education track courses" or by creating more stringent graduation requirements; 

• developing or revising curricula based on labor market information and employer input; 

• teaching employability skills in the classroom, including SCANS-based skills that were 
developed with a board of business and industry representatives; 

• teaching technical skills in the classroom, through the incorporation of industry- 
recognized technical standards into the curricula; and 

• informing students, employers, and teachers about legal and safety requirements for the 
work environment. 



Connecting Activities 

As was the case with school-based learning, LPs used a variety of connecting activities, 
including: 

• creating direct links between employers and teachers by hiring a teacher-employer liaison 
or offering teachers opportunities to spend time in the workplace (teacher 
intemships/extemships); and 

• offering business leaders the opportunity to work directly in schools through programs 
such as "Principal for a Day" or by helping with curriculum development. 
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Work-Based Learning 

One of the key provisions of the STWOA that sets it apart from other school improvement efforts 
is the emphasis on work-based learning opportunities. Work-based learning opportunities 
include: work experience opportunities; job training and work experience coordinated with 
learning in school-based programs that are relevant to students’ career major choices and lead to 
the award of skill certificates; instruction and activities in general workplace competencies; and 
broad instruction in all aspects of the industry (School-to-Work Glossary of Terms, 1996). LPs 
often connect work-based learning opportunities to other local programs such as Tech Prep and 
ROC/P. Overall, qualitative data indicate that LPs are using a variety of approaches to helping 
students learn about the real-life demands of the workplace, including internships, job 
shadowing, and community service. To facilitate participation and support among employers, 
labor organizations, public and private agencies, teachers, counselors, and other staff members, 
LPs are: 

• providing professional development for stakeholders; 

• hiring personnel to serve as liaisons between employers and schools; and 

• supporting the collaboration of different stakeholders as they create curricula. 

Access for All Students 

In their efforts to provide access to STC for all students, LPs commonly built on programs that 
already target specific groups that serve a diverse group of students. Quarterly reports indicate 
that a number of LPs have coordinated efforts with programs such as Tech Prep and Jobs for 
Youth to recruit community partners and find placements for work-based learning opportunities. 

Summary of Early Quantitative Findings 



Major findings from the LP Survey in 1998 provide some quantitative data concerning LPs’ 
organizational structure and implementation activities. This survey had an 82% response rate. 
Survey data provide a picture of STC during the early stages of California’s implementation of 
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STWOA, and allow some comparisons to data at the national level. Major findings from the LP 
survey are summarized below. 

• Almost one-half (47.8%) of LPs are led by education institutions, but data suggest that 
almost one-third (30.4%) are led by more than one type of organization, indicating a high 
level of collaboration between different groups within the LPs. 

• There is a fairly even distribution of high schools, middle schools, and elementary 
schools in LPs, and every responding LP included at least one post-secondary institution. 

• Private sector firms have a much higher level of participation in LPs than trade 
organizations, labor unions, or Chambers of Commerce. 

• With regard to career awareness and development, career inventories, individual career 
counseling, and activities integrated into academic and vocational classes were the most 
frequently reported STC components at the high school level. These activities were also 
offered by approximately one-third of post-secondary institutions. 

• LPs are promoting access to all students by using a variety of strategies, including: 
requiring that representatives of targeted groups (e.g., English Language Learners; 
students with disabilities) serve on their governing boards; offering funds for special 
activities related to the needs of these groups; training individuals from the business 
community; and using targeted promotional materials and special career guidance. 

• LPs are using a variety of strategies to focus high school students on career majors. The 
most common approach to career majors in California involves providing students with 
written course sequences to help them make course selections. 

• There is not a standard concept of curriculum integration across LPs. A little over one- 
half of LPs respondents (56%) developed career-related units or projects and even fewer 
(25% to 33%) used strategies such as academic-vocational team teaching or the use of 
state-developed applied curriculum materials. 

• Collaboration between secondary and post- secondary institutions most commonly 
involves sharing employer networks and contacts (48.9%) or sharing labor market 
information (48.6%). Moreover, close to half of California high schools allow students 
dual enrollment in high school and college. 

• With regard to work-based learning, the most prevalent activities in high schools in 
California partnerships include work site visits and job shadowing (about 43%). High 
intensity activities such as workplace mentoring and unpaid internships are offered far 
less frequently (in approximately 10% and 32% of high schools, respectively). 
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Summary of Local Partnership Sustainability Efforts 

In quarterly reports to the state, LPs described a variety of strategies for sustaining STC efforts. 
These strategies include: 

• recruiting representatives from key stakeholder groups (including local businesses, 
community-based organizations, post-secondary institutions, and parents), and creating 
systems for communication and collaboration; 

• securing ongoing resources and long-time commitments from local employers, 
particularly work-based opportunities, as well as finding alternative funding sources; 

• linking to community-based organizations such as Chambers of Commerce, County 
Education Offices, Private Industry Councils, and Workforce Development Councils to 
share information about their services and discuss ways the organizations can work with 
LPs; 

• providing professional development for stakeholders about the goals and strategies of 
STC, appropriate curriculum and instructional techniques, and administrative aspects of 
STC management such as grant writing; and 

• aligning efforts with other career-related programs, such as ROP/C, Tech Prep, and 
JTPA. 

Implications of Findings for the Evaluation 

As demonstrated above, we now have large amounts of data about STC in California. However, 
they are based on few common measures or reporting formats; are primarily qualitative; and 
focus on processes/plans implemented by LPs rather than student outcomes. Given that LPs are, 
for the most part, building systems and programs, collecting these data is appropriate. However, 
at present, data about STC in California do not provide information that is of the highest interest 
to funders and the Legislature (i.e., student performance data). Nor do they allow us to directly 
answer key research questions. Other findings about existing data on California's STC efforts to 
date include: 

• The diversity of STC implementation in California suggests that both attitudinal and 
performance measures are needed to measure the full impact of STC on students, 
teachers, employers, and systems. 

• On a statewide basis, California maintains school-level, not student-level data about 
student performance. 

• As evidenced in individual LP STC implementation plans, there is a lack of common 
concepts and vocabulary, exacerbating evaluation difficulties associated with a lack of 
common measures across LPs. 

• California's response rate to STW-related surveys, and therefore the availability of key 
comparable quantitative data, is notably low. Specifically, California's response rate on 
the 1998-99 Progress Measures Survey is 53.3%, with only 24 of 45 LPs responding. 
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• Available data on STC implementation in California lack specificity with respect to either 
quality or intensity of STC activities. 



In summary, the currently available data do not allow for strong inferences about the status of 
STC in California relative to the research questions of sustainability and student preparation, or 
how well LPs are meeting their goals. However, while existing data have important limitations, 
particularly with respect to STC impact and student outcomes, meaningful information on STC 
process and implementation can be gleaned from them. 

Overall, our analyses of recent STC implementation, data quality, and data availability indicate 
that upcoming evaluation activities must strike a balance between (1) canvassing LP activities 
across the state to achieve a broad understanding of California's STC progress and impact and (2) 
delving deeper into a more limited number of LPs to better understand systemic change and 
sustainability of STC. Additional implications include the following: 

1. Diverse STC implementation strategies suggest the need for some flexibility in data 
collection as well as common protocols/measures for Phase II evaluation activities. 

2. Diverse implementation strategies aiming for impact at multiple levels (student, teacher, 
employer, local system, state system) necessitate evaluation at a variety of levels. 

3. Because many STC activities aim to change students' attitudes as well as skills and 
knowledge, attitudinal measures are appropriate in conjunction with performance and 
other outcome measures. 

4. Seeking correlations between STC activities and changes in student performance or 
attitudes in places where STC activities have been minimal (e.g., one Career Day over the 
course of a year) will yield minimally useful information. Therefore, evaluation activities 
need to focus on LPs where STC activities have been substantial. 

5. While there has been limited implementation of complex school-based and work-based 
activities, we must focus on these activities, as they have the greatest chance for 
measurable student impact. 

6. Given that existing data offer little to no comparability, standardized, well-defined 
indicators must be developed. Similarly, standardized, well-defined data-collection 
instruments and methods are necessary. 

7. Given that many LPs, particularly new ones, are attempting to build systems, rather than 
isolated programs, evaluation of system change is critical. 

California STC Evaluation Plan 
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Our attempt in this White Paper to define the baseline status of STC activities in California 
highlights the complexity of the California STC landscape, as well as the significant challenges 
inherent to evaluating California's progress and potential for sustaining a statewide STC system. 
At the same time, the great variety of the STC activities across the state described in this White 
Paper suggests that there is significant fodder for the Phase II evaluation of the impact and 
sustainability of STC in California. The following are four guiding principles that underlie the 
evaluation plan: 

• The research questions dictate the focus of the evaluation. 

• Triangulation of multiple methods and data sources will help ensure the validity of the 
evaluation findings. 

• Stakeholders need regular opportunities and multiple avenues to inform the evaluation 
process. 

• The evaluation study must successfully balance the need to minimize burden on LPs with 
achieving comparability in data and data gathering. 

Evaluation Methodology 

The evaluation methodology consists of four major categories of evaluation methods: a 
systematic review of data from extant measures, project-developed survey research (i.e., 
employer/labor survey and project-specific survey of LP leadership), a comparative analysis of 
California’s STC effort to those of other select states, and longitudinal case studies. Both 
quantitative and qualitative evaluation research techniques will be used across the four 
methodologies. The first three evaluation methods are aimed primarily at cultivating breadth of 
information. In contrast, the longitudinal case studies will provide the opportunity for detailed, 
in-depth information to better understand complex issues, such as the impact of STC 
involvement on student outcomes, the contribution of STC to systemic change, and the 
sustainability of STC efforts. 

1. Review of data from extant measures 

Just as extant information was analyzed and synthesized for the White Paper, project staff will 
continue to examine existing sources of data for evidence related to this evaluation's questions of 
interest. Such external data sources include the Progress Measures Survey and the Local 
Partnership Survey. We will analyze and synthesize the information from extant sources across 
years to make comparisons and look for growth or other trends of STC activity within and across 
LPs. 
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2. Project-developed survey of LP leadership and survey of employers/labor organizations 

The leadership of all California LPs will be surveyed with a project-developed instrument at key 
points during Phase II. The purpose of this survey is to collect targeted information from all LPs 
about their STC activities that is not available through existing information sources. It will be 
administered by mail twice during Phase II: Spring 2001 and Spring 2002. 

A survey of employer and labor organizations was first administered in May 2000. This four- 
page survey asked employers and labor organizations about their awareness of STC, the nature of 
their participation in STC activities, their working relationships with the LPs, their perceptions 
about the benefits and challenges of STC, and their willingness to make long-term commitments 
to STC. This same instrument will be administered to employers and labor organizations again 
during the Spring of 2001 in order to obtain more comprehensive information about employer 
and labor involvement in STC. 

One option for the second administration of the employer/labor survey, given the resources of 
this project, is to cast the second survey administration as a follow-up to the previous survey 
administration. Specifically, we are proposing a two-pronged survey approach for employers: (1) 
a re-survey of the actual respondents to the first survey and (2) a survey of another random 
sample of employers from the existing project database of employers. For labor organizations, 
we propose: (1) a re-survey of respondents to the first survey and (2) a survey of additional labor 
organizations identified by the LPs involved in the case studies. 

3. A comparative analysis of California ’s STC effort with those of other states 

In an effort to go beyond California data to understand STC in California, we will examine the 
performance of three exemplar states representing STC environments and strategies particularly 
relevant to California (Oregon, Michigan, and Florida) and will track the progress of STW in 
these three states. These comparisons should illuminate how California is performing relative to 
other successful states. The comparisons will center on key indicators from Progress Measures 
and LP Surveys that address system growth, sustainability, and availability/spread of STC 
student experiences. Also, interviews will be conducted with administrators in each state, 
focusing on STC rollout strategies and activities. 

4. Longitudinal case studies 
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Executive Summary 



The strategy is to select 15 to 18 LPs for case studies. As was specified in the RPP for this 
evaluation study, $2.5 million will be distributed to LPs to conduct these studies. The overall 
sample should include a diverse group of LPs, representing different geographical locales, sizes, 
and implementation strategies. Consideration in the selection process will be given to the quality 
of the LPs' previous evaluation efforts, their commitment to using state-developed instruments, 
and the LPs' capacity to provide data, particularly student performance data. The case studies 
design will involve extensive use of both quantitative and qualitative techniques. With respect to 
quantitative methods, LPs will collect and analyze the following types of empirical data: 
numbers and types of schools, teachers, and students participating in STC activities; and student 
enrollment, attendance, and performance data. 

Project staff will be responsible for providing orientation, training, and ongoing 
monitoring/technical assistance to selected LPs on the case study design and data collection. 
Project staff also will be responsible for the development of all common evaluation 
instrumentation, monitoring the data-collection efforts of the LPs, and summarizing and 
reporting the aggregate results from the case studies to the lAP. Individual LPs will be 
responsible for conducting the case studies for their respective sites, including collecting the 
data, conducting first-order data analysis (e.g., generating descriptive statistics), and reporting 
results (e.g., data summaries, evaluation reports). 

Reporting of Evaluation Results 

The results of the evaluation study will be presented in two separate reports: an interim report 
(due June 2001) and a final report (due June 2002). The interim report will summarize the 
evaluation research results through Spring 2001, serving to foreshadow the summative findings 
of the comprehensive final report. 



er|c 



18 



XIV 



Introduction 



Evaluation Overview 

In January 2000, California’s governor, in collaboration with the Superintendent of Public 
Instruction and the Chancellor of the California Community Colleges, contracted with WestEd 
and MPR Associates to design and conduct a statewide evaluation of School-to-Career (STC) in 
California. This evaluation is required under provisions of the School-to-Work Opportunities Act 
of 1994 (STWOA), which awarded funds to California to support the development of its STC 
system. Findings from the evaluation will be a key information source for the many federal, state, 
and local stakeholders who are working to improve our education system. 

The overarching research questions listed in California’s STC evaluation Request for Proposals 
(RFP) are: 

• How has STC affected student preparation for post-secondary education and career 
entry? 

• To what degree and in what ways has STC contributed to systemic change? 

• Have STC principles penetrated the community deeply enough to be sustainable? 



California’s implementation and evaluation of STC depends heavily on the work undertaken by 
Local Partnerships (LPs), which have received the vast majority of STWOA funds. STWOA 
defines a LP as “a local entity that is responsible for local School-to-Work Opportunities 
programs” (Section 4(1) of STWOA). LPs are required to include employers; representatives of 
local educational agencies, post-secondary institutions, and organized labor; local educators; and 
students. There are 59 LPs in 1999-00; 45 state implemented subgrants and 14 federally- funded 
Urban/Rural Opportunities Grants (UROGs). 

To meet federal requirements for STWOA funding, the evaluation must focus on the LPs’ efforts 
to implement STC, and their ability to: 

• provide students with “higher-quality academic and work site instruction than that 
previously provided to students”; 

• prepare all students “for viable careers. . . and with a clear understanding of the 
education requirements to reach their career goals”; 

• involve multiple stakeholders (parents, employers, etc.); and 
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• integrate “state and national skill standards, articulation, and all other applicable criteria 
into local STC implementation efforts.”^ 



The evaluation RFP specifies that the evaluation be conducted in two distinct phases. Phase I 
includes the following components: 

• a research synthesis of information on STC in California presented in the form of a 
White Paper; 

• a design framework informed by the White Paper, for LPs to conduct in-depth 
longitudinal case studies; 

• recommendations for distributing funds to LPs for these case studies; 

• a survey of employers and labor organizations; and 

• creation of a statewide evaluation plan. 

Phase II of the evaluation involves implementing the evaluation plan, conducting a second 
survey of employers and labor organizations, and completing interim and final reports about the 
status and influence of STC in California. 



Purpose of the White Paper 

The main goal of this White Paper is to lay the foundation for the evaluation to be conducted in 
Phase II. The Paper focuses on the contextual factors that will impact the evaluation design, data- 
collection efforts being undertaken by LPs and external evaluators, and availability of data 
relevant to research questions presented in the RFP. Consistent with the RFP, the White Paper 
also serves the following specific purposes: 

• provides baseline data on the extent to which STC components have been implemented 
in California; 

• provides information on the extent to which data are currently being collected and the 
collection methods used, that will address the overarching research questions; 

• identifies proposed research questions and the need for subsequent evaluation study 
design components; 

• determines feasibility factors and contextual issues that will impact the evaluation study 
design and implementation; and 



^ California’s School-to-Career Request for Applications (2000), p. 3. 
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• proposes a statewide evaluation strategy. 

Using a variety of sources — including LP applications for STWOA funds, LPs’ Quarterly 
Reports, data from two federally-sponsored longitudinal evaluations of STWOA implementation, 
and existing evaluations conducted for LPs (see Appendix A for a full listing of sources) — 
WestEd and MPR examined the status of STC in California. This Paper discusses these findings 
and emphasizes how they relate to designing a feasible evaluation strategy for Phase II. 

Organization of the White Paper 

The White Paper contains four major sections and is organized to provide readers with 
information about the national, state, and local contexts in which STC is being implemented. 

Section I provides a brief historical view of School-to-Work (STW), describing the underlying 
national goals of the STWOA and how this legislation relates to other school reform efforts. This 
section also discusses California’s efforts related to the STWOA and briefly summarizes work 
conducted through California Partnership Academies and the Carl D. Perkins Vocational and 
Technical Education Act (Perkins III), including its Tech Prep provisions. 

The second section of the Paper provides readers with contextual information about STW 
implementation across the nation. It gives an overview of the main findings of three national 
studies of STW, with special emphasis on describing the institutional participants, career 
awareness activities, efforts to promote access for all students, implementation of career majors, 
curriculum integration, and work-based learning. 

The third section narrows the focus to California, specifically describing the goals and strategies 
that LPs in California are using as they implement STC. It provides a rich description of how 
LPs are working to reach the goals set forth by the STWOA and to build their STC initiatives, 
giving illustrative examples of efforts related to school-based learning, work-based learning, 
connecting activities, and access for all students. Then, early quantitative findings from the Local 
Partnership Survey (1997-1998), conducted by Mathematica Policy Research for the Planning 
and Evaluation Service of the U.S. Department of Education, are presented. This discussion 
covers six major areas; partnership participants; career awareness and development; promoting 
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student access; career majors; curriculum integration; and work-based learning. Finally, this 
section discusses LPs’ efforts to build a sustainable STC system. 

The fourth section of the White Paper summarizes what is known, and not known, about 
California’s STC system from existing data. It then presents implications of these findings for the 
evaluation design. 

The Paper concludes with a comprehensive plan for evaluating California’s STC system. 
Specifically, it describes the guiding principles for the evaluation, the major components and 
core data elements, and how results of the evaluation will be reported. 
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This section of the White Paper provides an historical overview of the evolution of STW, and 
brief discussions of related national and state legislation and reform efforts. This contextual 
information is helpful in understanding current efforts in California as the state works to create 
and sustain its STC system. The Smith-Hughes Act, passed by the United States Congress in- 
1917, was designed to support vocational education programs that were generally aimed at 
helping low-performing students prepare for occupations that did not traditionally require 
advanced degrees. Since that landmark legislation, efforts related to preparing students for the 
workforce have changed dramatically at the national and state levels. 



From Vocational Education to STW 

Educational studies released in the 1980s, such as A Nation at Risk (U.S. Department of 
Education, 1983) and The Forgotten Half: Non-College Youth in America (William T. Grant 
Foundation, 1988), reflected a growing perception that secondary schools were not preparing 
students adequately for work or post-secondary education. According to a report by the 
Department of Labor Secretary’s Commission on Achieving Necessary Skills (SCANS), released 
in 1991, fewer than half the students graduating from high school in the United States had the 
skills and knowledge they would need for productive work in the labor market. Moreover, 
colleges and universities reported that students entering their institutions had low levels of 
academic knowledge. These reports, combined with shortages of qualified entry-level employees 
in the business community, led to changes in the vocational education model. Early vocational 
education was directed toward students who wanted to work in occupations that typically did not 
require advanced degrees. High school students were placed in tracks depending on their 
academic abilities. Generally, high-performing students who were preparing for college took few 
or no courses related to vocations, and low-performing students placed in the vocational 
education track did not take courses that would enable them to enter college. 

In the 1990s, the goals of vocational education initiatives began to change and a new STW model 
emerged. In contrast to federally-funded vocational education initiatives supported by the Smith- 
Hughes Act, current STW initiatives blur the distinction between academic and vocational 
education. STW initiatives are now designed to develop academic and work-based skills for all 
students. While there are many different approaches to STW, all have at their core the goal of 
improving academic achievement while at the same time providing students with marketable. 
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work-based skills. Another common element that distinguishes vocational education from STW 
initiatives is the use of a formal, three-pronged approach in STW: students engage in school- 
based and work-based activities, and schools and their community partners provide vehicles to 
connect the two. STW initiatives are required to promote the involvement of multiple 
stakeholders in the education process. These include employers, labor organizations, teachers, 
parents, and community-based organizations. Moreover, STW programs often emphasize the 
integration of industry-related and academic standards into classroom instruction. 

Major Legislation Supporting STW 

Carl D. Perkins Vocational and Applied Technology Education Act 

The Carl D. Perkins Vocational and Applied Technology Education Act of 1990 (Perkins II) 
helped create the new vision that ultimately became the STW model. This legislation' promotes 
“reform, innovation, and continuous improvement in vocational and technical education” 
(Summary of Public Law 105-332) so students will meet rigorous academic and industry 
standards and be prepared for post-secondary education and high-skill, high-wage careers. 
Perkins II emphasizes strong linkages between secondary and post-secondary education; 
professional development for teachers, counselors, and administrators; and the active 
involvement in education of parents and employers. Perkins III, authorized in 1998, goes even 
further in supporting these types of connections by developing core indicators of performance 
and promoting integrated “one-stop” education and workforce development systems at the state 
and local levels. This legislation also established increased accountability for states, programs, 
and schools by setting state levels of performance and subsequent penalties for non-compliance 
with core indicators. 

Tech Prep 

One aspect of the Perkins legislation that further supports the evolution of vocational education 
to STW is funds set aside under Title III (E), Tech Prep, which was enacted in 1990. Title III of 
Perkins provided funding for special grants for high schools to work with two-year colleges or 
apprenticeship programs to create a coherent sequence of study for students that combines 
academic and vocational courses. Tech Prep strengthened the role community colleges play in 
STW programs, and enhanced articulation between secondary and post-secondary institutions. 
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Typical Tech Prep programs begin in students’ third year of high school, and continue until 
students have finished two years of post-secondary study and obtained an associate degree in a 
technical field of work. Tech Prep, reauthorized as part of Perkins III in 1998, has seven key 
components: 

• the development of articulation agreements between secondary and post-secondary 
participants; 

• the design and implementation of an educational program with a common core of 
courses leading to required proficiency in mathematics, science, communications, and 
technologies which leads to an associate degree or certificate in a specific career field; 

• the inclusion of Tech Prep program curricula appropriate to the needs of secondary and 
post-secondary students; 

• in-service training for teachers; 

• training programs for counselors; 

• equal access for all students; and 

• preparatory services that assist all populations. 



The School-to-Work Opportunities Act 

In 1994, the STWOA was passed by Congress for the purpose of underwriting the initial costs of 
establishing statewide STW systems in all states. The Act has many goals similar to Perkins in 
that it emphasizes the integration of academic and vocational education and rigorous standards 
for all students. The STWOA specifies that state STW systems should be integrated with systems 
developed under Goals 2000: Educate America Act and the National Skill Standards Act of 
1994. These systems should be part of a comprehensive educational reform strategy designed to: 

• enable students to earn portable credentials; 

• help all students reach high academic and occupational standards; 

• prepare students for high-skill, high-wage careers; and 

• increase students’ opportunities for further education. 



The STWOA indicates that state STW systems should involve close collaboration between 
educators and employers. These connections will facilitate using workplaces as learning 
environments, and promote the formation of LPs designed to link schools, private and public 
employers, labor organizations, government, community-based organizations, parents, students, 
and other stakeholders. 
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One-time, five-year competitive grants to states were authorized under STWOA. States are 
expected to identify a range of resources to sustain the STW system when federal funds sunset in 
2001. 



Eight states received implementation grants in 1994 (totaling $100 million); 19 additional states 
received funding in 1995 (totaling $245 million). In 1996 and 1997, 10 states were awarded 
grants totaling $350 million (California received $21,900,000). By 1998, all states had been 
awarded multi-year STW implementation grants. In addition to state grants, Urban/Rural 
Opportunities Grants (UROGs) were awarded to LPs focusing on youth in high-poverty areas as 
well as other Local Partnership Grants from the Departments of Education and Labor. 

States are allowed considerable leeway in designing their STW systems so that each state can 
work to meet its specific needs and challenges. However, all systems are expected to include 
certain core elements'. 

• integrating school-based learning (e.g., career awareness and exploration, career majors, 
career counseling) and work-based learning (e.g., work experience, workplace 
mentoring, instruction in general workplace competencies, job shadowing); 

• connecting activities (e.g., providing students with school site mentors; providing 
technical assistance to employers; training teachers, staff, and other key stakeholders; 
encouraging active participation of employers); 

• integrating academic and occupational learning; 

• establishing effective linkages between secondary and post-secondary education; 

• establishing career majors (coherent sequences of courses or fields of study that prepare 
students for a first job); 

• providing opportunities for students to learn all aspects of the industry in which they are 
interested (e.g., technical and production skills, planning, management, finances); and 

• ensuring equal access for all students to the full range of program components. 

As a result of this flexibility, state profiles describing their STW approaches show a wide variety 
of implementation strategies. Regardless of the implementation approach used, however, 
STWOA encourages states to incorporate STW into their overarching education reform 
strategies. 

Other National Reform Efforts 

The U.S. Department of Education also initiated related programs aimed specifically at high 
schools, such as the Comprehensive School Reform Demonstration developed within OERI 




26 



8 



I. AN HISTORICAL OVERVIEW OF SCHOOL-TO-CAREER IN CALIFORNIA 



(Office of Educational Research and Improvement) and the New American High Schools 
(established by the Office of Vocational and Adult Education). Like Perkins III and STWOA, 
these reforms include integration of academic and vocational curricula and the establishment of 
partnerships with employers. Moreover, influential high school reform networks, such as the 
Coalition of Essential Schools and the “High Schools that Work” project, support many of the 
principles underlying the STW movement because STW goals are consistent with the stated 
principles of these reform movements. Specifically, STW as a reform effort can provide; 

• meaningful context for students’ academic work across disciplines; 

• high expectations with real-world standards; 

• opportunities for all students to be actively engaged in the learning process; and 

• authentic learning environments that motivate all students to learn and to pursue 
additional education. 



California’s STC Plan 

Soon after passage of the STWOA, California received a federal developmental grant that 
provided the foundation for creating the state’s STC plan. This includes establishing the STC 
Task Force (later the Advisory Council), the Interagency Partnership, and developing 
implementation grant structures to fund LPs. 

California's STC Vision 

California’s vision for STC is that all students “will develop a love of learning, and be fully 
prepared for rewarding high wage, high skill careers; productive citizenship; and personal and 
professional growth” (1996 Application, p. 1). In creating the work plan, a strong effort was 
made to reach “full consensus” of “all of the principal policy and political interests in education 
reform” (Work Plan, p. 2). 

The initial principles on which California’s STC system was built are: 

The system is an integral part of broad education reform, workforce preparation, and 
economic development efforts. 

Education reform is of primary importance. 

All students must have access to an STC system. 

Industry plays a central role in the emerging STC system. 
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• California’s STC system has a strong and clear partnership structure. 



Award of Development Grant — 1994 

To develop the state’s STC plan, California’s governor appointed a 27-member STC Task Force. 
The Task Force was charged with developing and presenting a comprehensive state plan to the 
governor for approval, which would build on and integrate existing STC programs that were 
currently operating in the state, including Tech Prep and Partnership Academies. With the 
primary goal of developing a plan that would increase the efficacy of California’s educational 
system, while better preparing students for a competitive and changing economy, the Task Force 
presented an initial plan with the following key recommendations: 

• Career Pathways, Standards, and Certification 

Improve K-12 and community college education systems by better orienting students to 
career opportunities and providing opportunities for learning in work site and community 
settings. High Schools should be accountable for providing students with the core skills and 
knowledge they need to enter universities and the workforce. 

• Local Partnership Development and Demonstration 

Ensure statewide implementation through LPs where organizational and governance 
arrangements are made at the local level. Initial implementation grants will be awarded to 
LPs to establish a series of comprehensive demonstration sites, selection based on the 
comprehensive nature of their proposed system.^ 



^ California’s School-To-Career State Plan, Chapter II Executive Summary, pg. 2. 
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• The Roles of Business and Labor 

Demand greater involvement by employers and labor unions and provide systems and 
incentives to sustain that involvement. 

• Collaborative Administration and Implementation 

Develop a structure where the administration and implementation of the state’s plan are 
guided by representative advisory bodies and/or committees whose members include subject 
matter experts and representatives of key interests. 

To assist in managing the state plan, the Interagency Partnership (lAP) was established with key 
representatives from the California Department of Education, the Chancellor’s Office of the 
California Community Colleges, and the Employment Development Department. The role of the 
lAP is to provide state-level management including vision and leadership; marketing and 
communication support and strategies; program and performance oversight; evaluation 
assistance; fiscal oversight; and day-to-day management and system coordination. Creation of 
the lAP increased state-level collaboration among educational and workforce development 
agencies. 

Implementation Grant — 1996 

In 1996, California received a state implementation grant of $21,900,00. (In total, California may 
receive up to $130 million over a five-year period.'*) 



Basic goals for California’s STC system include: 

• improving connections of K-12 schools and community colleges with universities, 
business, and labor; 

• increasing career awareness activities for all K-8 students; 

• organizing instruction at the secondary level around career pathways with integrated 
academic and vocational/technical education; 

• creating a certification system that incorporates occupational and professional 
requirements; 

• developing policies regarding incentives for business and labor participation in the 
implementation of STC to encourage employer involvement; and 

• instituting a performance-based system to monitor STC implementation. 



Based on the lessons learned during the development phase, California’s governor made 
additional appointments to the STC Task Force, creating the STC Advisory Council. Prior to 



California School-To-Career Statewide Evaluation RFP, (1999). 
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launching a statewide plan, the Advisory Council formed a marketing advisory group to assist in 
determining how best to develop a statewide marketing plan. A poll of California parents’ 
knowledge of STC was conducted to learn how and where to focus communication efforts. 
Based on the survey data and with the assistance of regional media collaboratives, a statewide 
marketing plan was designed to increase the general population’s STC awareness through: PSAs, 
a general brochure, and by establishing “800” lines to field questions. Strategies were also 
developed with Junior Achievement. 



To assist in the complex issues surrounding implementation, the Advisory Council established 
various committees and advisory groups representing the interests of education and industry as 
well as ensuring the comprehensive implementation of the state plan. Four committees were 
appointed: 

• Education Issues and Practices, to advise on key program development and 
implementation issues. 

• Student Assessment and Certification, to make recommendations on how best to 
coordinate and integrate STC principles into the statewide assessment system and assist 
in linking STC to state adopted standards and assessments. 

• Employer and Labor Participation, to develop models and systems to increase and sustain 
industry and labor participation as well as focus on addressing barriers to involvement by 
industry and labor. 

• Systems Evaluation and Accountability, to deal with issues related to the long-term 
success and viability of the statewide system and design the system evaluation. The 
Committee coordinates an evaluation plan with an evaluation consultant, assists in 
developing an evaluation design and identifying data elements needed, provides funding 
for longitudinal case studies, and assists in developing a statewide MIS system and data- 
collection strategy for K-I2 to eventually provide follow-up information on student’s 
post-secondary outcomes. 



Local Partnerships 

Since the inception of California’s STC plan, it was determined that LPs would be at the heart of 
the state’s comprehensive STC system. To distribute funds to local STC advocates, the state was 
divided into twelve regions aligned closely with the K-12 Superintendents, Goals 2000, and 
Community College regions.^ Initially, the state-funded LPs in defined regions to ensure that 
California’s geographic and social diversity was represented; however, future funding has since 
been on a competitive basis. (Independent of state-funded LPs, approximately 14 federally- 



’ State of California Implementation Grant Continuation Application Matrix - DRAFT May 2000. 




30 



12 



I. AN HISTORICAL OVERVIEW OF SCHOOL-TO-CAREER IN CALIFORNIA 



funded UROGs have been awarded in California.) Grant selection criteria for LPs include 
demonstration of equal opportunity and access by all students and require strategies for 
sustainability and systemic implementation. 

Over the past four years, California has awarded over $94 million to 45 LPs (15 in 1997, 37 in 
1998, 45 in 1999 including 7 federal and 38 state sites) with yearly awards ranging from $96,000 
to $1.5 million.® More than 90% of California’s student population are served by this 
comprehensive initiative.^ 

Support Strategies 

To provide additional support to LPs, additional associations were established. With the 
assistance of K-12 schools and Community Colleges, The Regional Coordinators Network was 
established to assist the state in its rollout strategy. The role of the Network is to develop plans to 
coordinate outreach to funded and unfunded sites and to provide technical assistance to LPs as 
needed. The Northern and Southern California STC Practitioner Coalition was established to 
provide collaboration opportunities among the LPs in order to foster integrated regional system 
implementation. In an effort to disseminate information regarding useful tools and effective 
practices learned during this implementation period, an RFP has been established for a resource- 
clearing house to provide information on-line or in hard copy. 

The Advisory Council also established several competitive awards focused on strategic planning 
and implementation. A contract was awarded to CSU Hayward to integrate STC into the state’s 
pre-service teacher prep program. Ten teams were formed throughout the state to integrate STC 
pedagogy and practice into higher education pre-service programs. The Coast Community 
College District/Orange County received funding to develop aligned curriculum and provide 
training to LPs regarding work-based learning. A Technical Assistance Peer-to-Peer grant was 
awarded to Tulare County Office of Education to provide LPs with strategic planning assistance 
and establish Tulare as a demonstration evaluation site. A technical assistance grant was also 
awarded to CSU Bakersfield to focus technical assistance to Challenge Districts and all LPs. 



® Schedule of Funding to Local Partnerships, Attachment 5. 

^ Sehool-to-Career Significant Interagency Accomplishments to Date, July 1999. 
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In collaboration with the EDD’s Labor Market Information Division, the California Department 
of Education (CDE) has begun to identify business and industry sectors that will be the 
foundation of Career Pathways and industry skill standards statewide. The goal of the Career 
Pathways effort is to identify 15 economic sectors (14 have been identified for approval by the 
Advisory Council) that will be used as a framework for infusing career-related information and 
experiences into the curriculum, and assist in program planning and career guidance information 
specific to each of the state’s 12 regions. 

As demonstrated by the state’s efforts in supporting a strong network of LPs as well as 
developing a state-level support system, California’s STC implementation strategy is a “bottom- 
up” as well as “top-down” approach. The goal of this purposeful approach is to continue building 
on the successes of the local efforts. Thirty-eight state-funded LPs will continue to receive 
funding through 2000 and 23 through 2001. Committees initially established for the 
implementation period continue to strategize ways in which successful efforts can be sustained 
after federal STWOA funding ceases in 2001. 

Major STC Efforts in California 

Various initiatives currently exist in California that were created in the 1990s to provide career 
awareness and employment-related skills. Some of these initiatives were specifically designed as 
opportunities for job training, such as those funded by the Job Training Partnership Act (JTPA), 
which provided services for economically disadvantaged adults and youth, dislocated workers, 
and others who faced substantial employment barriers. (The Workforce Investment Act of 1998 
(WIA) rewrites current federal statutes governing programs — including the JTPA, replacing the 
statutes with streamlined and more flexible components of workforce development systems.)* 

Other California initiatives, such as California Partnership Academies, have broader aims and 
emphasize the integration of school- and work-based learning, and promote curriculum 
integration, K-16 articulation, and the involvement of employers, labor organizations, schools, 
parents, students, and other key stakeholders. These initiatives are closely aligned with national 
activities that support STW and also support California’s own STC effort. 



* Workforce Investment Act of 1998 (On-line), Available www.telamon.org/wia/index 
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These key initiatives, which preceded California’s successful application for STWOA funds in 
1996, will be described below. These initiatives provided the foundation for California’s current 
STC efforts. 

California Partnership Academies 

California Partnership Academies, which are prototypes for what are called “career academies” 
throughout the nation, are funded by the state. California Partnership Academies are a strong 
component of California’s STC system. Over the last decade, the number of California 
Partnership Academies has been growing steadily. 

The roots of California Partnership Academies can be traced to reform efforts in Philadelphia, 
where a number of career academies were created to prepare high school students for occupations 
that do not typically require a bachelor’s degree, such as automotive mechanics or electronics. In 
the early 1980s, two academies were created on the San Francisco peninsula. These programs 
targeted at-risk students in danger of dropping out of high school and were funded by community 
organizations, foundation grants, and business partnerships, not by the state. 

Documented success of the career academies in these two schools led to the establishment of 
state-supported academies throughout California in diverse fields such as law, business, tourism, 
and finance. While the original academies in Philadelphia were designed as vocational education 
programs, the goal of California Partnership Academies is to offer students high-level vocational 
training in selected occupations, as well as to provide them with the option of going to college. 
Although academies differ on a number of dimensions, they share three basic features, including: 

• small learning communities, where a cluster of students share several classes each year 
and have the same teachers for at least two years. Typically, an academy program 
operates as a school- within-a-school; 

• a combination of a career theme and college preparatory curriculum. Academic courses 
meet high school graduation and college entrance requirements. The curriculum includes 
work-based learning opportunities; and 

• strong partnerships with employers. 

Evaluations of career academies suggest that these programs can be effective in reducing dropout 
rates, improving school attendance and increasing graduation rates. There is also some evidence 
that these programs have led to better work attendance and job performance by graduates. 
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In 1994, the STWOA explicitly listed career academies as one of the “promising practices” for 
preparing students for both work and further education. Moreover, the career academy approach 
was also supported by the CDE in its publication. Second to None: A Vision of the New 
California High School (1992), which provides direction for restructuring high schools. The 
report recommends that students choose 

“an organized program around a specific focus that combines academic, applied 
academic and field experiences. These program majors can be organized 
according to themes built around career fields ... or integrated academic 
disciplines. . . Many integrated programs are designed to meet college entrance 
requirements, while also providing students with career-related technical and 
practical skills” {Second to None, p. 21). 

With the underlying support of the STWOA and the CDE, the number of career academies has 
increased in California, and in some instances has been the basis for district-wide reform. For 
example, in Oakland, a district-wide policy was adopted in 1996 that would eventually lead to all 
students enrolling in career academies in 10"' to 12"' grades. In some cities in California, 
extensive networks of academies now exist. A 1996 report to the CDE shows that the state- 
funded academy population was fairly stable between 1992-93 and 1995-96, with 45 academies 
and enrollments of approximately 5,500 students in grades 10-12 in each of these four years. 
However, by 1998, California had over 200 academies, and projections suggest that the number 
of academies will continue to increase. 

Tech Prep 

Like career academies. Tech Prep programs are designed to integrate academic and vocational 
education. Moreover, they both focus on enhancing academic performance, improving 
articulation between secondary and post-secondary institutions, and helping students to be better 
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prepared to enter the workforce. Finally, both include programs in a wide variety of areas such as 
engineering technology, business education, agriculture, industrial technology, and health 
careers. 

From 1993 to 1997, about $12 million per year was allocated for Tech Prep efforts in California, 
with the following goals: 

• establishing a formal structure for articulation within each college district service area 
for the development of Tech Prep programs; 

• developing Tech Prep Education programs to provide a formal course of study for all 
students; 

• expanding current articulation efforts to all segments of secondary/higher education 
within the state; and 

• increasing statewide awareness of the benefits of program participation. 

To meet these goals, three program components were created for Tech Prep: local Tech Prep 
consortia, statewide resource centers, and special statewide projects. By 1998, 83 consortia were 
operational in all of the state’s community college districts, and six Tech Prep Resource Centers 
were created. Three statewide projects were also created related to evaluation, outreach and 
guidance, and student follow-up. In 1998, Tech Prep was offered in over half of the state’s high 
schools. 

An evaluation of Tech Prep in California (Evaluation and Training Institute, 1997-1998) 
identified factors that contributed to “effective” Tech Prep consortia, and found that these 
consortia had the following characteristics: 

• a clear understanding of the goals and purposes of the different reforms; 

• joint committees, including representation from all stakeholder groups, to integrate 
existing reform efforts; and 

• one year of planning prior to implementing Tech Prep. 
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Other major findings related to Tech Prep include: 

• Tech Prep in California “lacks a formal, consistent statewide definition,” which has led to 
“general confusion concerning the goals and objectives of Tech Prep, and it has also 
allowed each consortium to define and implement Tech Prep in its own particular way”®. 

• Articulation agreements between secondary and post-secondary institutions exist in all 
consortia in some form, but there is typically no formal procedure for tracking students as 
they move from one educational level to another. 

• Curriculum development as a direct result of Tech Prep has been minimal. 

• Tech Prep does not explicitly require work experience or work-based learning. STWOA 
funds have been used by some Tech Prep consortia for work-based learning. 



This evaluation also examined the interrelationships among Tech Prep in California, federally- 
funded LPs, and Academies. Evaluators report: 

“The greatest barrier to the successful integration of reform efforts throughout the 
state has been the recurring ‘turf battles between members of different interest 
groups.'®. . . Lack of a clear understanding of the goals and purpose of Tech Prep 
and STC has hindered efforts to integrate.”" 

California’s Regional Occupational Centers and Programs (ROC/Ps) 

The mission of California’s ROC/Ps is to “provide quality career training programs which 
contribute to students’ educational achievement and to the economic development of California 
through career guidance, employment training, job development, and other educational support 
services for high school students and adults.” There are currently 72 ROC/Ps operating in 
California. Predating the STWOA of 1 994, ROC/Ps have accomplished the following: 

• offered more than 100 different career preparation courses in Health, Consumer 
Homemaking/Home Economics, Business Office/Marketing, Industrial and Technology 
Education, and Agriculture; 

• provided competency-based job training for over 450,000 high school and adult 
students; 

• served a diverse range of students by working in conjunction with school districts and 
articulating with community colleges and state universities; and 



’ Evaluation and Training Institute, 1997-1998, p 15. 
'® Ibid, p 1 
" Ibid, p 27 
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• collaborated with other federal and state agencies and programs, including STW, 
Welfare to Work, Job Training Partnership Act, the Department of Rehabilitation, and 
Private Industry Councils.'^ 



California’s One-Stop Career Center System 

In January 1997, California received an $8 million implementation grant from the U.S. 
Department of Labor to develop and implement a statewide One-Stop Career Center System. Six 
months later, California awarded 18 grants totaling nearly $5 million to local One-Stop 
partnerships throughout California. Known as California WorkNet, this system of One-Stop 
partnerships and services supports the integration of education, job training, and employment 
programs, services, and information, to ensure they are delivered in a comprehensive, customer- 
focused manner. Services provided by this system are directed towards job, education and 
training seekers, as well as employers. 

Summary 

California's STC environment includes many important components (a statewide system of LPs, 
a state STC plan, coordinated governance at the state level, involvement of diverse stakeholders, 
and substantial pre-STWOA STC initiatives, such as Tech Prep and Partnership Academies). 
Furthermore, California has kept pace with and built on federal STC legislation and initiatives. 
However, STC and its national context, which is important both for California's STC success and 
for its STC evaluation, are changing rapidly as federal funding sunsets. 



California Association of Regional Occupational Centers and Programs, (On-line), Available www.carocp.org. 
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National Progress on Implementing a STW Effort 

To assess California’s STC effort, it is important to place the state’s accomplishments in a 
national context. Understanding how the STW system has developed across the country since 
1994 will ultimately allow for a comparison between California’s system and national results. 
Reviewing the national studies of STW implementation will allow us to answer some of the 
following questions; 

• In what ways is California’s experience similar to the national experience? In what ways 
is it different? 

• What have been the national challenges and accomplishments and how do these compare 
to California’s experience? 

• What data are collected by national studies, and how should this influence the evidence 
needed to evaluate STC in California? 

• What does the wide variability of STW implementation across the nation suggest about 
what we can expect of STC in California? 

This section of the Paper gives an overview of the main findings of three national studies of 
STW, with special emphasis on describing the institutional participants, career awareness 
activities, efforts to promote access for all students, and implementation of career majors, 
curriculum integration, and work-based learning. 

Funded by the National STW Office, three major studies — the National Evaluation of STW 
Implementation (hereafter referred to as the National Evaluation), the STW Progress Measures 
(hereafter referred to as Progress Measures), and STW in the 1990s: A Look at Programs and 
Practices in American High Schools (hereafter referred to as NLSY), have collected and reported 
information on how students, schools, districts, employers, and communities are responding to 
the call for more comprehensive, coordinated school reform that links schooling to the world of 
work. (See Appendix B for an abstract of the studies). These studies show that there has been a 
surge of activity encouraged by the STWOA during the 1990s. Assessing the extent to which this 
has substantially changed the relationship between schools and work or the awareness of students 
about the world of work is a more difficult question, one that is unfortunately not yet definitively 
answerable with the available data. 
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Key Findings from the Three National Studies 
Governance/institutional participants 

As encouraged by the STWOA, LPs involve a wide variety of educational institutions and 
employers. 

• By the fall of 1997, 34 states had formed 1,152 LPs, covering 83% of their secondary 
school districts and 90% of all students of high school age in the grantee states (National 
Evaluation, p. 29). 

• Approximately 55% of the schools involved with LPs are elementary, 20% are 
middle/junior high, 20% are secondary schools, and 6% are other grade-level 
configurations (Progress Measures, p. iii). 

• Nearly 178,000 private, public, and nonprofit employers are engaged in partnership 
activities, 109,000 of which provide work-based learning positions to students (Progress 
Measures, p. 21 and p. 23). 

• Nearly 2,600 post-secondary institutions — two- and four-year, public and private, as well 
as proprietary schools — work with LPs. Of this total, 48% of these are two-year 
institutions, 36% are four-year, 7% are private career schools, and 10% are other types. 
Only 4% of LPs report no linkage with any post-secondary institution. (Progress 
Measures, p. 6). 



Although STW was intended to integrate education and workforce development efforts, STW is 
an enterprise typically led by educational institutions in most states. Of the 37 funded states, 22 
chose to locate the state STW office in the department of education. With regards to the 15 other 
states: two elected to administer STW through a department of labor or commerce, eight granted 
oversight to a workforce development board, and five decided to locate the STW office within 
the Governor or Lieutenant Governor’s office (Erlichson and Van Horn, 1999, p. 5). Similarly, at 
the local level, 54% of LPs are led by a school or school district, 7% by a post-secondary 
institution, and only 5% by an employer or business group. The other 34% fall into categories of 
either “other” or “none named.”(National Evaluation, p. 37). In sum, school districts and high 
schools typically lead the STW effort at the local level, with the following levels of collaboration 
with other key groups: employers (most extensively), post-secondary institutions (less 
extensively), and organized labor (least extensively). 

Career development 

The national evaluations indicate that most states and LPs emphasize broad career development 
over other components of STW implementation (National Evaluation, p. xv). Career 
development (the process through which students understand their place in the world of work) 
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involves a continuum of activities: career awareness, career exploration, work exposure, career 
guidance, and post-secondary and/or employment planning. Often beginning as early as the 
elementary grades, students become aware of the skills critical to success in the world of work. 



Figure 1 depicts the growth in availability of different categories of career development activities 
from the 1996-97 school year to the 1997-98 school year. Specifically, it shows that higher 
percentages of high schools are introducing practices like career awareness courses, career 
development units in other courses, individual career plans, interest inventories, and scheduled 
use of career centers (National Evaluation, p. 60). Other key observations about career 
development activities across the nation include the following:'^ 

• In 1998, about four out of five seniors participated in interest inventories, employer talks 
at schools, and/or work-readiness classes, and two out of three participated in a job- 
shadow or work site tour at some point during high school (Ibid, p. 62). 

• In 1997, about half of elementary schools in LPs reported offering significant career 
information within the curriculum. One year later, this increased to about two-thirds of 
elementary schools (Progress Measures, p. 10). 

• In 1998, 68% of middle/junior high schools in reporting LPs provided structured 
exploration activities, such as individualized learning plans linked to career pathways 
offered in high school. This compares with 59% in 1997 (Ibid, p. II). 

• The NLSY study demonstrates the variability in types of career awareness activities in 
public high schools across the nation. College awareness, interest inventories, and career 
awareness are the most commonly offered career guidance services, with more than 90% 
of schools saying they offer these activities. Less prevalent, but still fairly common, are 
job site visits and skill inventories (at 73%). About half of public high schools create 
individualized career plans for students, and 44% of schools involve parents in these 
career plans. 



Data Source: U.S. Department of Labor, Bureau of Labor Studies, National Longitudinal Survey of Youth, 1996- 
97, Survey of School Administrators; U.S. Department of Education, National Center for Education Statistics, 
Common Core of Data Survey, 1993-94. 
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Figure 1. Growing Availability of Career Development Activities 

[National Evaluation Fig. III.l, p. 60] 
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Data Souree: STW Loeal Partnerships Survey, fall 1996 and fall 1997, Mathematiea Poliey Researeh, Inc. 
Note: The latest data from LPs on work site activities are for school year 1996-1997. 



Promoting student access 

Promoting student access involves ensuring that students from a broad range of backgrounds and 
circumstances — including both males and females; disadvantaged students; students from all 
racial, ethnic, and cultural backgrounds; students with and without disabilities or English 
proficiency; school dropouts and migrants; and the academically talented — are included in STW 
activities. The National Evaluation found that LPs have been relatively successful at promoting 
STW for all students, although participation rates among subgroups of students vary to some 
extent. Based on information from the National Evaluation’s eight in-depth study states. Table 1 
shows a breakdown of student participation in three STW components (comprehensive career 
development, workplace activity linked to school, and career-related academies) by students’ 
gender, race, class rank, high school attendance, enrollment in “basic college prep” courses, and 
post-secondary education enrollment. 
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Table 1. Student Participation in Three STW Components in the Eight In-Depth Study 
States, by Student Background, School Performance and Program: Class of 1996 

[National Evaluation, Tables IV. 1 & IV.2, pp. 104 & 108] 



Percent of Students Participating During High School 



Group 


Comprehensive Career 
Development 


Workplace Activity 
Linked to School 


Career-Related 

Academies* 




All students 


63.3 


16.1 


11.7 




Gender 


- 








Male 


62.1 


13.6* 


10.7 




Female 


64.4 


18.4* 


12.7 




Race/Ethnicity 










African-American 


67.0 


22.7* 


13.1 




Latino 


60.6 


14.9* 


17.2 




White/other 


63.2 


15.2* 


11.0 




Cumulative High School Class Rank 








Top quartile 


63.6 


17.0 


13.2* 




Middle two quartiles 


64.6 


16.3 


13.5* 




Bottom quartile 


63.1 


15.6 


9.5* 




Cumulative High School Attendance 








Top quartile 


68.4 


14.7 


17.0* 




Middle two quartiles 


64.4 


17.3 


16.7* 




Bottom quartile 


63.2 


14.5 


9.4* 




Basic College Prep*"* 




' 




Yes 


61.4 


14.7 


11.7 




No 


65.7 


17.4 


13.2 




Post-secondary Education Outcome 








Four-year college 


62.7 


14.3* 


12.7 




Two-year college 


64.6 


18.7* 


9.9 




No college 


62.9 


17.1* 


. 12.0 



Date Source: Transcripts of the class of 1996 and STW survey of 12‘** graders, spring 1996, Mathematica Policy 
Research, Inc. 

* Differences among the groups are significant at the .05 level, two-tailed test. 

♦Career Academies are high school programs that are usually “schools-within-schools” (i.e., smaller administrative 
units operating within larger schools) that are occupationally focused. 



Basic College Prep is four years of English, two years of a foreign language, and three years of math, science, and 
social studies. 
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As shown in Table 1 : 

• The activities promoted in the STWOA are engaging students at all academic 
performance levels. For example, student participation in comprehensive career 
development or workplace activities that are linked to school is unrelated to student 
achievement (as measured by cumulative high school class rank in the eight in-depth 
study states). However, students in the bottom quartile of their class are slightly less 
likely than their peers to participate in career-related Academies (9.5% versus about 13% 
for the higher quartiles). 

• Students who completed a college preparatory curriculum are just as likely as students 
who had not completed such a curriculum to participate in comprehensive career 
development, workplace activities linked to school, and career-related Academies. 

• Students who enrolled in a four-year college are slightly less likely than their peers who 
either did not enroll in college or enrolled in a two-year institution, to participate in a 
workplace activity linked to school. However, they are very similar to their peers with 
respect to comprehensive career development and career-related Academies. 

• Female students are involved at higher rates than male students in workplace experiences 
linked to school. In the eight in-depth study states, 18.4% of females are involved in such 
activities, versus 13.6% of males (National Evaluation, p. 108, table IV. 2). 

• African-American students also have higher than average participation rates in workplace 
activities (22.7%), relative to students from other racial-ethnic backgrounds (14.9% of 
Latinos and 15.2% of whites). 



STW efforts aim to help all students develop the skills to explore alternatives and select careers. 
One key element in this exploration process is helping students identify broad areas of career 
interest. Focusing on this interest will help them choose advanced and elective high school 
courses and make decisions about post-secondary education. Examining change in the percent of 
students who defined a broad career interest while in high school reveals that in certain respects, 
STW is reaching a broad mix of students. In other respects, disparities remain. Figure 2 shows 
the growth from 1996 to 1998 of the percentage of seniors in the National Evaluation who 
selected career interest areas and were involved in STW activities by urbanicity, post-secondary 
plans, and race. As shown by this figure: 

• In 1996, rural students were more likely than students in urban or suburban schools to 
select a career area. By 1 998, rural students were still the most likely to choose a career 
area, but the disparity between urban/suburban and rural had shrunk considerably. For 
example, in 1996 there was a 16-point gap in the participation rates of urban and rural 
students. By 1998, this gap had shrank to three points. 

• Students in 1996 who had no plans for college were less likely to have chosen a career 
area than their peers who had planned to go to a two- or four-year college. By 1998, the 
participation rate of students with no college plans had increased to a level of parity with 
students with college aspirations. 
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• With respect to race-ethnicity, however, disparities have widened in the eight in-depth 
study states, as African-American students continue to choose career areas to plan for at 
higher rates than whites or Hispanics. In 1996, the white-African- American participation 
difference was eight percentage points, and the African-American/Hispanic participation 
difference was 13 percentage points. These gaps were wider in 1998: the white/Attcan- 
American participation gap was 17 percentage points, and the African- 
American/Hispanic gap was 24 points. 
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Figure 2. Growth in Selection of Career Area and Involvement by Urbanicity, by Students’ 
Post-secondary Plans, and by Race* *® 

[National Evaluation, Figures IV.3, IV.5, & IV.6, pp. 1 16, 1 19 & 120] 
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Data Source: STW 12*-grade survey, spring 1996 and spring 1998, Mathematica Policy Research, Inc. 

* Differences among the groups are significant at the .05 level, two-tailed test. 
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Career majors 

Career majors are a key element of the STWOA. Participation in career majors signifies that 
students are pursuing a coherent sequence of courses that integrates academic and occupational 
learning, emphasizing all aspects of an industry, coordination of school-based and work-based 
learning, and, ideally, preparation for further education and/or training {STW Glossary of Terms, 
1996). While a career path is more of an opportunity to introduce students to career options, a 
career major begins when the student makes a focused career selection. Ideally in California, the 
career-awareness-to-selection continuum is first introduced in elementary schools where students 
gain an understanding of the many career opportunities available. By middle school students 
begin to consider their individual interests and aptitudes in narrowing their career plans and by 
high school a career major may be selected and required career coursework completed. The 
learning continuum continues through post-secondary coursework until preparation is completed 
in readiness for career entry. 

Because of the considerable flexibility granted to states, LPs, and schools, career majors rarely 
adhere closely to the definition of the STWOA. As actually implemented, career majors are often 
defined by three features that schools adopt to varying degrees (National Evaluation, p. 66): 

• Students choose broad career areas after some exposure to information about careers. 

• Career pathway brochures are published to help students choose elective courses that fit 
best with their career areas. 

• Schools offer defined programs of study for students who choose a particular career area, 
and all students who choose a particular career area take the same course of study. 



On the whole, career majors are given less emphasis than exploratory activities and career 
guidance (National Evaluation, p. 134). Site visits have suggested that most schools have 
structured programs for only one or two careers, and that typically no more than several dozen 
students are involved per school (Ibid). Reflecting the emphasis on career exploration, schools 
are more likely to use career pathways as tools for guidance counselors to use when helping 
students choose electives. 
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Figure 3 shows data from the National Evaluation on the prevalence of career majors in the 
1997-98 school year. These data indicate the following: 

• 46% of high schools report that they have implemented career majors in one form or 
another. However, little is indicated about the characteristics of these career majors. 

• 42% of high schools in LPs report offering career majors based on written plans. 

• Facets that require more planning and staff support are less common. More structured 
approaches that involve grouping students into the same academic or vocational classes 
are present in 29% of LP high schools. 

• Only 6% of schools require all students to take a sequence of courses aligned with their 
career majors. (Data suggest that these are mostly vocational schools.) 

• About one in four high schools offer career majors linked with work-based learning 
options. 



Figure 3. Definition of Prevalence of Career Majors in School Year 1997-1998 

[National Evaluation, Figure III.3, p. 68] (Percent of LP High Schools) 



Career Majors 
Implemented 
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Curriculum integration 

Integrated curricula involve academic and occupational/career subject matter taught in a manner 
that emphasizes relationships between and among different disciplines {STW Glossary of Terms, 
1996). Integration can take the form of coordinated scheduling across subjects, introducing more 
academic content into vocational classes, incorporating more exaniples from the world of work 
into academic classes, or comprehensive programs where all instruction centers on career major 
themes. The proponents of curriculum integration view it “as a way to ensure that instruction in 
technical or practical skills includes a strong theoretical base and challenges students 
intellectually and that academic instruction emphasizes the ability to apply theoretical 
knowledge” (National Evaluation, p. 71). 



The most commonly reported changes related to curriculum integration were in the curricula of 
specific courses. For example: 

• More than half of high schools in LPs offer applied academic courses (where applied and 
project-oriented instructional materials and methods are used in academic classes) and 
this percentage has been increasing (54% of schools in 1996-97, 58% in 1997-98) 
(National Evaluation, p. 72). 

• Almost one half of schools have a course that teaches all aspects of an industry in a 
vocational course, and this percentage has also been increasing (42% in 1996-97, 46% in 
1997-98). 

• 56% of secondary students in LPs were exposed to work-related curricula in 1998, and 
about one in three was exposed to coursework that integrates academic and vocational 
curricula. The percentages of students involved in these two activities is almost 
unchanged, indicating that while schools are reporting increases in opportunities for 
integrated curricula, students may be unwilling or unable to take advantage of them. 

Site visits conducted for the National Evaluation indicate that “there are limits to the support for 
curriculum integration and constraints on its progress” (National Evaluation, p. 73). In many 
schools, academic teachers are less interested than vocational teachers in integrated curriculum. 
Academic teachers often worry that incorporating practical or hands-on learning will detract 
from the more traditionally-defined academic skills important for doing well on standardized 
tests, college admission, and success in college. Other factors include: teachers have limited 
common planning time across departments; applied coursework does not always meet college 
admission criteria; and time for curriculum development is severely constrained (Ibid). 
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Work-based learning 

The STWOA envisioned work-based learning as an integral part of broadening student 
awareness of the relevance of academic subjects and the expectations of the workplace. In 
practice, work-based learning takes many forms in U.S. high schools and funded LPs, ranging 
from one-time visits to work sites — with very little connection to what students are learning in 
school — ^to paid youth apprenticeships that are the culmination of a coherent sequence of 
coursework. Not surprisingly, the more intensive types of work-based learning opportunities are 
the least common. Figure 4 shows the percentage of public high schools that reported offering 
various types of work-experience activities in 1996. These data indicate: 

• The most common work-based STW activities are work-based learning during school 
hours and paid work experience (both at just over one-half of high schools) and job 
shadowing (offered by 43% of high schools in 1996). 

• Less common are internships, mentorships, apprenticeships, and school-based enterprises 
(all at 20-25% of high schools). 

These figures do not tell us whether these placements are linked to school-based learning, 
however. In 1997, half of high schools in funded LPs reported offering work-based learning 
connected to integrated curriculum. By 1998, this figure had increased to 63% of high schools in 
LPs (Progress Measures, p. 13). Unfortunately, the percentage of students who take advantage of 
these opportunities is rather low. In 1998, only 16% of secondary students in funded LPs 
participated in a work-based learning opportunity connected to integrated curriculum (Ibid, p. 
14). 



Figure 4. Percentage of Schools Offering Work-Based STW Activities that Focus on Work 
Experiences, by Type of Activity: 1996 [NLSY, Figure 1, p.l 1] 
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Data Source: U.S. Department of Labor, Bureau of Labor Statistics, National Longitudinal Survey of Youth, 1996- 
1997, Survey of School Administrators; U.S. Department of Education, National Center for Education Statistics, 
Common Core of Data Survey, 1993-94. 
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Figure 5 shows the percentage of high schools in funded LPs that offer various work-based 
learning opportunities and the percentages of high school students that take part in these 
activities. While many high schools offer work-based learning opportunities, Figure 5 suggests 
that only small percentages of students take advantage of these opportunities.'® 

Figure 5. Percentage of Secondary Schools and Secondary School Students in Reporting 
LPs Participating in Work-Based Learning Activities: June 1998 

[Progress Measures, Figures 6 & 7, pp. 15 & 17] 
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Data Source; MPR Associates, Inc., Progress Measures Survey, July 1, 1997-June 30, 1998. 

The National Evaluation confirms the finding that “brief work site activities for career exposure 
are most prevalent” (National Evaluation, p. 83). Because work site visits and job shadowing 
experiences can last only a few hours, school staff, employers, and students find them easy to 
integrate into the traditional school routine. More time-intensive opportunities require greater 
levels of commitment from students, school staff, and employers, and are understandably less 
common. Another important finding is that paid or unpaid positions obtained through school 

The participation rates should be considered conservative estimates because of the difficulty schools face in 
compiling data on work-based learning placements. Because of the complex and time-consuming nature of 
tracking student placements, it is quite likely that a certain number of placements are never reported. 
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provide students with greater learning opportunities in several respects (National Evaluation, p. 
89-90): 

• Compared to positions not obtained through school, students work in a wider variety of 
industries. Student work experience not obtained through school is concentrated in retail 
and food service establishments. 

• More time on the job is spent training and students have more access to career 
information. 

• Students receive more feedback on performance. 

• Links between school and workplace are more common, which helps reinforce the link 
between school and work. 

Unfortunately, these opportunities are relatively rare. Only about one quarter of paid or unpaid 
positions that students held were obtained through school in 1998 (National Evaluation, p. 86). 
(See Appendix F for Major Issues For Future National Research.) 

Summary 

The three national studies of STW show considerable progress in only a few years toward 
implementing key elements of the STWOA. However, many schools are providing only some 
parts of the STW school-improvement strategy, and few students participate in a comprehensive 
career-focused high school learning experience. The National Evaluation indicates that schools 
have a long way to go to integrate comprehensive career development activities, career-related 
academics, and paid or unpaid work experience that is linked to learning in the classroom — only 
about 3% of students have been involved in all three of these approaches. Moreover, although 
critical goals of the STW movement are to support high academic standards and produce positive 
student outcomes, it is too early to identify the role that STW is playing in achieving these goals 
(National Evaluation, p. 139). In most states, implementing STW and raising standards and 
accountability are two important school improvement efforts that have been operating 
independently of each other. As the next section about STC in California details, California has 
much in common with the nation as a whole, as well as significant differences. 
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Whereas the previous section of the White Paper focuses on the national context for STW, this 
section focuses attention squarely on California and the work of the LPs. It begins with a 
description of the goals and strategies that LPs are using to implement STC, as well as 
descriptions of their efforts related to these goals. A summary of these qualitative findings is then 
provided. Next, early quantitative findings from the Local Partnership Survey (1997-98) are 
described. The section concludes with a discussion of LP efforts in California specifically related 
to sustainability. 

The State and LP Goals/Strategies 

As part of examining LP efforts toward creating STC systems and toward attaining goals, it is 
appropriate to review the parameters in which the LPs operated. After receiving STWOA funds 
in 1996, California disseminated a Request for Applications (RFA) to identify LPs eligible to 
receive money to support STC implementation. To qualify for funding, LPs were required to 
show that the system would be accessible to all students and would not result in tracking. Other 
required elements include the integration of academic and vocational curricula in career pathway 
programs, as well as work site learning, career guidance, and connecting activities. LPs were 
expected to indicate how they would leverage, redirect, and reallocate local existing resources, 
and how funds would be used to support educational reform through systemic change, rather than 
supporting or developing categorical programs. 

In the RFA, the state did not specify an implementation model for LPs to follow. Rather, LPs 
were given the freedom to identify successful approaches and to develop systems to meet their 
own needs. However, LPs were required to complete a Performance Matrix as part of the 
application process, which became the work plan for those LPs that received funding. The 
Performance Matrix focuses on four major goals; 

• Comprehensive Local STC System; 

• Quality, Effectiveness, and Scope of the LP; 

• Participation of All Students; and 

• Management Plan. 

Within each of these four goals, the state specified “System Elements,” such as “Work-based 
Learning” which are listed as elements of the Comprehensive Local STC System (see Appendix 
C). LPs were expected to address each of these System Elements at some point within the five- 
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year funding period. LPs were required to provide the following details for each of these System 
Elements: 

• specific activities and strategies planned by the LP; 

• timelines for completing these activities; 

• outcomes (defined by the state as “specific measures of achievement” that “define the 
desired results”); and 

• budgeted amounts allocated to the activities, and the organization or person responsible 
for the activity/strategy and outcome. 

Because the state gave LPs freedom in choosing their activities and strategies, it is not surprising 
that LPs used a wide variety of approaches for addressing each System Element. An analysis of 
Performance Matrices of LPs was conducted, and examples of different activities and strategies 
for each Goal and System Element are listed in Appendix D. As this analysis shows, the 
activities and strategies LPs chose to use varied considerably. For example, for the System 
Element “integrated academic and applied education,” LPs conducted staff development, gave 
teachers mini-grants, held district conferences, developed mentoring relationships between 
schools, and created career pathways — to cite just a few examples. 

LP Efforts Related to Goals 

The best available data source about LP implementation efforts that was completed by large 
numbers of LPs is the quarterly reports LPs completed for the state. The following discussion of 
STC activities is based primarily on qualitative information fi"om those narrative quarterly 
reports, as well as a small number of in-depth STC evaluations conducted for individual LPs. It 
is organized around four main categories of STC goals that are the basis of the quarterly reports: 
school-based learning, work-based learning, connecting activities, and access for all students. 

LPs’ quarterly reports summarize their strategies and activities and provide a rich and detailed 
inside perspective on what they are doing to implement and sustain STC. However, these reports 
generally are not consistent with each other in style or format, and they do not include 
quantitative or comparable measures. Consequently, fi"om these reports we carmot draw 
inferences about activity levels within or across LPs, nor can we discuss progress toward 
achieving goals. We are, however, able to provide illustrative examples of how STC initiatives 
are being implemented in the state of California. Please note that the use of examples from 
particular LPs generally reflects the quality and amount of data about various LPs provided to the 
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authors of this report. The LPs mentioned below are not the only LPs implementing exemplary 
practices. 

We begin by reviewing implementation activities and then move on to sustainability efforts. 
Clearly, it is important to understand that the two are closely aligned. Many efforts that work 
toward sustaining STC are also integral to successful implementation. For example, longer-term 
goals, such as comprehensive curriculum change, come about through short-term iimovations in 
school-based or work-based learning and enhanced professional development. Similarly, 
coimecting activities that help stakeholders collaborate in the short term may lead to long-term 
coimections with other school and community programs and will help sustain the STC effort. 

School-Based Learning 

School-based learning can be implemented in many ways. According to definitions provided by 
the National STW Office, the school-based learning component of an STC system should include 
activities related to career awareness, career exploration, and counseling programs at the earliest 
possible age, but no later than the seventh grade. Also, it should incorporate career major 
selection no later than a student’s junior year of high school. Schools should provide a program 
of study that meets the academic standards of the state and meets post-secondary and skills 
certificate requirements. As an overall goal, they should also have a program of instruction and 
curriculum that integrates academic and vocational learning in all aspects of an industry. Finally, 
the school-based learning component of STC should provide students with easier transitions into 
post-secondary education programs, training programs, and the transfer of students between 
educational and training programs {STW Glossary of Terms, 1996). 

As demonstrated throughout the next section, California LPs report a multitude of school-based 
activities to reach these overall educational objectives. LPs have been active in supporting the 
provision of career counseling and career exploration at all levels. LPs encourage the 
development of curricula that include relevant labor market data to support these standards. 

Career exploration 

School districts involved in STC are encouraging a progressive sequence of career and college 
awareness activities that begin in the elementary grades and continue past high school. At the 
elementary level, the focus is primarily on exposure to different careers by bringing in guest 
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speakers, engaging in role-playing and arranging field trips to local businesses. In middle school, 
career exploration takes on greater focus through more intensive individual counseling and the 
development of student portfolios, building upon the career exploration activities at the 
elementary school. In high school, schools are providing in-depth career exploration through 
more extensive career guidance and integrated career related courses. Some LPs are actively 
facilitating linking the activities from elementary school to secondary and post-secondary 
through career pathways and career clusters. 

• Vandenberg Middle School teachers organized a two-week career exploration unit with 
their 8th grade class. Students took an aptitude survey and completed the “Pathways to 
the World of Careers” survey to discover their “pathways” of interest. They completed 
a personal survey comparing their values to possible occupations. Based on these 
activities, students chose two careers to research. Then, students investigated reasons 
for attending college, and developed both personal and professional goals. In the second 
part of this project, students were randomly assigned a “life” — a job, an income, marital 
status, number of children and pets. Based on this information, students created a 
budget, worked through tax forms, analyzed and graphed how they spend their time, 
and determined what they need to get a job. In another class, students prepared a cover 
letter, resume, and job application. They also had the opportunity to hear guest speakers 
talk about the education and skills necessary for various professions. Finally, students 
used their career research to register for high school courses. High school counselors 
noted these students were better prepared to select courses according to their interests. 

• The East San Gabriel Valley LP actively ensures all school districts offer career 
awareness activities, labor market information, and opportunities for students to explore 
interests. The LP promotes a progressive range of activities beginning in elementary 
schools with career awareness activities, and continuing through post-secondary career 
preparation. Several high schools have instituted career pathways. Students in 8th grade 
design four-year education plans with STC in mind and have access to guidance 
software. 

• A variety of classroom presentations take place at schools in the Imperial Valley 
Business/Education Coalition. In 1998, they reached an audience of 8,983 students. 
Presentations covered opportunities in law enforcement and fire departments as well as 
information about requirements for careers in state prison facilities, the medical field, 
civil engineering, music, retail/sales, and business/accounting. They also had 4,660 
students participate in field trips to area businesses, community fairs, and festivals, and 
1 1,517 students from all grade levels attended career fairs in 1998. 

• A third grade class in the Santa Barbara STC Consortium spent time creating a career 
magazine. Each student investigated a career of his or her choice. As part of the 
assignment, students invited and arranged for professionals in the community to come 
and speak to the class about their careers. All of the students prepared questions to ask 
each speaker and took notes. The information they obtained from these speakers was 
incorporated into their career magazine. 

• The North Coast STC Consortium engaged in several exploratory activities at each of 
its individual sites. A “Community Career Day” as part of “Parents as Experts Day” 
was established at the Sunny Brae Middle School. Also at the Sunny Brae Middle 
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School, they aligned career pathways with the elementary, middle, high and post- 
secondary schools and developed student portfolios. At the Jacoby Creek Elementary 
School, also within the North Coast Consortium, they implemented career shadowing, 
8th grade aptitude assessments, career-related videos, Internet exploration and electives 
focusing on occupations. 



Career counseling and guidance 

In-depth career counseling and guidance present a great challenge — ^the ratio of students to 
counselors in California is over 1,000 to 1. However, some schools report that counselors are 
becoming partners with teachers to connect career and college guidance activities with classroom 
work. Also, school personnel are capitalizing on the interest of their community partners to make 
career guidance a hands-on, practical activity. Through training and collaboration with local 
businesses, guidance counselors are becoming more aware of the employment opportunities and 
demands of the workplace. The role of the counselor is being redefined to include guidance on 
career choice and opportunity. Unfortunately, data from LPs about career counseling are limited. 

• When the Riverside County LP decided to review the needs of counselors in their area, 
it “became clear that counselors in high schools and middle schools know very little 
about career counseling. They are also unaware of the benefits of contextual and work- 
based learning strategies, and tend to place most of their students into courses that will 
meet college A-F requirements.” Following this observation, the Riverside County LP 
made efforts to in-service counselors on STC concepts and to encourage them to create 
plans for students that allow them the full range of career and post-secondary education 
options. 
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High student performance standards 

California’s LPs are working to meet their goal of raising standards for all students in several 
ways. For example, referring to all students, the Sacramento Regional STC Alliance notes that 
“establishing clear standards for the knowledge, skills, and abilities required of our workforce 
benefits industry in recruitment, hiring, training, and promotion and helps educators develop 
effective, relevant curriculum” (LEED-Sacramento Regional STC Alliance). High skill standards 
ensure relevance and quality assurance in education. 

High academic standards 

Raising student academic performance is a primary goal of California’s STC effort. Across the 
state, educators and their community partners are devising educational strategies that include 
STC learning opportunities while focusing on the growing concern for high student test scores 
and other positive student outcomes. 

• In the Riverside LP, 103 teachers were released to participate in four days of Integrated 
Curriculum training and developed curricula tied to rigorous academic and skill 
standards. 

• The minimum graduation standards across the Orange County CoalitionA^ision 20/20 
are being revised so they are aligned with new state standards and include STC 
components. Among specific areas being reviewed are the Math and Language Arts 
requirements and the inclusion of core competencies for career success and 
employability skills. 

Curriculum development 

One way LPs are working to implement school-based STC goals is through curriculum 
development. Some LPs are reevaluating their curricula and are revising them based on labor 
market information. By developing work-related curricula, LPs are creating institutional changes 
that will enhance sustainability. Some LPs have involved businesses and industry leaders in the 
curriculum development process, helping to ensure that students are exposed to realistic 
expectations and lessons. It will be important that teachers, educators and business partners 
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continue to be involved in curriculum development so current materials do not become obsolete 
with changing demands in the labor market. 

• In the State Center Consortium, 25 secondary schools have curriculum that is enhanced 
with references to the world of work, which is estimated to affect 4,338 students. Also, 
23 schools received training in curriculum integration/project-based instruction, 
including a total of 96 teachers, and potentially affecting 4,056 students. 

• Participants at the State Center Consortium also drafted a student activity that uses 
labor market information. Resources from the EDD and the existing career pathway 
labor market report to create a student activity were included in the unit. 

• Santa Barbara High School, within the Santa Barbara STC Consortium, has added 
several new components to its curriculum. Eight of the State Center Consortium Career 
Pathways are being introduced for students to choose from (Agriculture, Art and 
Communications, Business, Engineering Technology, Health Careers, Home 
Economics, Careers and Technology, Industrial and Technology, and Public and 
Human Services). They have also set up career academies, including the Multimedia 
and Art Design Academy and the Visual Art and Design Academy. These academies 
employ rigorous college-prep course work with technologically challenging, project- 
based learning. Also, new ROP classes have been instituted (Small Business 
Management, Computer Repair and Networking, and Horticulture and Geology) which 
lead to certificates and are articulated with the Santa Barbara City College. 



Employability skills in the classroom 

Some California schools are also raising the bar for technical education by connecting the 
curriculum to rigorous local or national technical standards. They are involving businesses and 
incorporating expectations for the workforce into curriculum and coursework. Several LPs have 
adapted the SCANS-based skills.'^ SCANS was developed with a board of business and industry 
representatives to promote employability skills by defining a three part foundation set of skills 
and a five part set of competency skills needed in the workplace. The foundation skills 
considered important for employability are basic skills, thinking skills, and personal qualities. 
The five workplace competency skills consist of the ability to use: resources, interpersonal skills. 



” In June, 1991, U.S. the Secretary’s Commission on Acquiring Necessary Skills (SCANS) produced What Work 
Requires of Schools which defined the workplace competencies and foundation skills required for effective job 
performance. 
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information, systems, and technology. Overall, LPs have been highly involved in recruiting 
business partners to actively participate in determining workplace needs and skills. 

• The Butte County STC Consortium provided intensive professional development 
focused on implementing employability skills in the classroom. Eighteen educators 
participated in a three-day workshop and a week-long summer internship. The 
internships were hosted by 26 different businesses. The workshops provided teachers 
with SCANS, work-based and school-based STC activities, and curriculum integration 
training. While interning, teachers were required to document daily examples of 
SCANS being demonstrated at work sites. They also had to conduct a business 
information interview, write an internship summary, and prepare a lesson plan using 
their experiences and training. 

• PacifiCare hosted a SCANS skills training for site teams in March of 1999. The day 
included a comparison of the work-based and post-work-based environments and the 
relationship between the conventional and SCANS classrooms. Time was allowed for 
team planning and application of strategies presented. The Orange County Coalition, 
Vision 20/20, also trained 110 teachers in integrating one or more performance-based 
employability skills during 1998-99, including Vital Link, Career Beginnings, and Tech 
Prep. 

• Charles Schwab is one of the most active businesses that provides comprehensive 
learning opportunities for students. School-to-Career at Charles Schwab begins in the 
earliest years of childhood education and continues through college and graduate 
school. As part of Schwab’s workforce development, the focus is to integrate firsthand 
business know-how into SCANS-based curricula through a variety of activities and 
collaborations. These include K-6 speaking engagements and work site field trips for 
students; job shadowing and in-school company volunteers in grades 7-10; and 
mentoring, curriculum development and part- and full-time employment opportunities 
for high school juniors through post-graduate students. 



Technical skills in the classroom 

Interest in raising students’ technical skills has moved in two directions: improving the rigor of 
vocational courses and giving all students broad exposure to the technical aspects of work 
through career-focused programs. Several LPs reported that they have incorporated industry- 
recognized technical standards into their vocational/technical curricula. These include standards 
that have been endorsed by the National Skill Standards Board, in fields such as 
banking/financial services, agriculture, and health services, and by national organizations such as 
the National Consortium for Health Science and Technology Education (NCHSTE), and the 
“Building Linkages” program which was initiated by the Office of Vocational and Adult 
Education (OVAE) in conjunction with the STW Opportunities Office. Others have fully 
integrated their STC and other career focused programs such as Tech Prep to maximize the value 
of both efforts for a broad range of students. Adopting industry-recognized skill standards, skills 
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certification, and connecting STC to successful Tech Prep programs will facilitate students’ 
transitions to work or higher levels of education. 

• California Works!, located in the Yuba Community College District, employs the Work 
Ready Certificate program. This program offers certificates to students who meet 
stringent criteria; they must demonstrate an excellent attendance record, attain a 
minimum grade point average, and have experience participating in work-based 
learning activities such as internships and job shadowing. Employers who have 
partnered with California Works! recognize this certificate as additional documentation 
for employment applications. Many students in this LP also have opportunities to earn 
occupational health and safety certificates. 

• The Sacramento Regional STC Alliance incorporates the California banking skill 
standards into their STC system. The banking standards are a joint project of the 
California Business Roundtable and the CDE. Educators and representatives from the 
banking industry have identified core skills necessary for all employees in the banking 
industry. They fall into three clusters: data and item processing, loan processing, and 
sales and service. 

• The 21st Century LP is engaged in a comprehensive program with Cisco Systems that 
will provide students with the necessary technical skills for careers involving high 
technical standards. Through an innovative partnership with Napa Valley Unified 
School District, Cisco Systems is preparing students for the demands and enormous 
opportunities of the information industry while creating a qualified talent pool for 
building and maintaining education networks. Cisco is an exciting STC initiative that 
teaches students how to design, implement, and troubleshoot computer networks in four 
semesters. Students who complete the course will be prepared to take an industry 
recognized certification exam (Cisco Certified Network Associate-CCNA) and 
compete for jobs in the Information Technology field. The curriculum also provides a 
solid foundation for students who plan to continue their education at the college level in 
highly sought after technical disciplines. 



Work-Based Learning 

One of the key provisions of the STWOA that sets it apart from other school improvement efforts 
is the emphasis on work-based learning opportunities. Work-based learning opportunities 
include: work experience opportunities; job training and work experience coordinated with 
learning in school-based programs that are relevant to students’ career major choices and lead to 
the award of skill certificates; instruction and activities in general workplace competencies; and 
broad instruction in all aspects of the industry {STW Glossary of Terms, 1996). Through STW, 
teachers also have the opportunity to learn directly from business and industry. Teachers 
participate in short-term work-based activities such as job shadowing, where they spend a limited 
time at the work place; or intemships/extemships where they spend an extended period of time, 
sometimes in paid positions. 
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California LPs are implementing STC strategies with a wide variety of work-related activities 
that reflect the diversity of their communities. Many of them are connected to other local 
programs, such as WIA, ROC/P, and Tech Prep, all of which have the goal of helping students 
learn about the real-life demands of the workplace. While providing more work-based 
opportunities, LPs are taking on a greater role in ensuring that these occur in a safe and legal 
learning environment. 

A broad range of work-based STC activities 

In the broad range of work-based STC activities implemented by LPs, students are able to 
develop their SCANS-based competencies, such as resource utilization, information skills, 
system skills, technological skills, and interpersonal skills. They are also able to clarify their 
educational and career goals and expectations. 

• The San Francisco headquarters of Charles Schwab and Company, Inc. demonstrates a 
deep commitment to education through a series of comprehensive STC activities. Each 
summer, students spend approximately 150 hours in challenging part-time, paid work 
assignments receiving school credit, adding to the over 1 00 students who work as part- 
time STC associates year-round. The STC experience is seen as a springboard 
encouraging post-secondary education and career options for students. Over 150 
teachers are encouraged to participate in shadowing, in-service training and externships 
to focus on high academic standards and expanded career skills. Additionally, the San 
Francisco headquarters leads the charge along with strategic business partners, such as 
Autodesk, Inc. and Bank of America, to work with key STC organizations in building a 
regional STC system. 

• The LP in Marin has established a comprehensive program of internships in a variety of 
careers ranging from teaching to video production and law. Local newspapers have 
highlighted these internships as exemplary activities in school reform. This positive 
attention from the press enhances outreach and awareness of STC initiatives. 
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• The Sacramento Regional STC Alliance offers some powerful opportunities in the 
banking and financial services, retail and marketing, and technology industries. 
Eighteen students were selected for competitive internships at Bank of America and 
River City Bank. The River City Bank was so impressed by student abilities that they 
opted to offer paid internships instead of the original job shadowing opportunity 
arrangements. Also, sixteen students were placed in six-week summer internships in 
high tech companies such as Intel, Montgomery Watson, and Comcast. 

• A General Motors (GM) project was created which falls within the Experience 
Education curriculum in the Santa Barbara STC Consortium. This is a special 
marketing program sponsored by GM and a local dealership. In 1999, fifteen students 
volunteered to participate in this project. They directly handled all aspects of a public 
relations campaign, i.e., budget, advertising, event coordination, etc. This cooperative 
education program clearly defines objectives for each student. The project allows for 
direct interaction with the employer and has an associate faculty member from the 
business department assigned to provide the work-based guidance and instruction. 



Learning in a safe and legal work environment 

As schools become a major contact point for coimecting students to the workplace, they must 
assume new kinds of responsibilities for ensuring that students have a safe and legal learning 
environment. Among LPs that report details about how they are providing a safe working 
environment, there is evidence of many different approaches. Some strictly post information 
about legal requirements. Others actively seek to ensure that participants are well aware of their 
rights. Some have focused primarily on students and school persoimel, while others have worked 
with students, educators, and employers. 

• At the Northeastern California STC Consortium, staff expect to provide teacher training 
on child labor laws to more than 135 teachers by the end of the 2000 school year. Since 
teachers are the contact point for coimecting students and employers, it is important that 
they are educated in what constitutes a safe and legal working environment. 

• A more detailed example of educating stakeholders came from the Solano County 
Office of Education, which has taken several steps: (1) This LP developed a booklet on 
legal work environments and labor laws and made it available to local employers; (2) 
Local employer partners and representatives from the local community college 
presented a training program to assist prospective employers about legal issues 
pertaining to youth labor; and (3) Educators are currently revising curricula to include 
material about safe and healthy work environments within career pathways. The Solano 
LP strategy involves educating not only the students and teachers but also the 
employers themselves. This type of outreach will be helpful not only in providing a safe 
and legal environment, but in recruiting a sample of small businesses as well. 

• The San Francisco LP provides legal information to employers, has arranged a resource 
to assist employers in designing appropriate internships, and also requires employers to 
sign insurance agreements to cover work-based learning opportunities. 
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• Finally, some LPs, such as the East San Gabriel Valley STC, institute requirements for 
employers to sign binding documents ensuring a safe and legal work environment. 
These are based in the agreements used in ROC/P programs in California. They hold 
the employer responsible for the safety of students. 



Conn ecting/Linking A ctivities 

Connecting both school-based learning and work-based learning is crucial to provide an overall 
contextual basis for learning and to ensure that STC activities do not occur in isolation. Students 
tend to learn more quickly when they are able to directly apply what they are learning in school 
to the work environment. Conversely, application of skills in the workplace illustrates the 
necessity of academic competence. Not all LPs directly reported on their methods for linking 
work-based learning and school-based learning components. LPs that reported on linking 
activities employ different strategies: hiring a teacher-employer liaison; engaging in teacher 
externships that expose teachers to the realities of work environments; educating business leaders 
through programs such as “Principal for a Day”; and involving teachers and the business 
community in curriculum development. Perhaps the most comprehensive strategy is the 
implementation of extensive projects that have been developed collaboratively with several 
stakeholders to incorporate both school-based learning and work-based learning. 

• One school district within the Horizons LP in San Bernardino County hosts an annual 
Youth Forum where key leaders from the community gather to address all six career 
pathways at the secondary institutions. Student representatives from all high schools 
attend the forum and return to their schools to share information with their peers in their 
English classes. 

• San Francisco STC helps to match work-based learning opportunities with high school 
and City College of San Francisco students through existing programs such as Travel & 
Tourism, Law Academy, Summer Training and Education Program (STEP), the 
Summer Youth Emploj^ent and Training Program (SYETP), and others. In 1999, 400 
students were placed in paid internships and over 600 were placed in categorical 
programs’ internships. STC has allowed these types of efforts to become centralized, 
even though they are accomplished through the collaboration of several different 
organizations. Furthermore, all pathways students are required to write and present 
reports discussing their work-based learning experience. They must identify what they 
have learned and assess how their experience improved their SCANS skills. Their 
employers receive copies of their reports and attend the presentations. 

• The Kern County Partnership has instituted comprehensive community service that 
incorporates other programs to ensure a high level of connecting activities. These 
activities involve community members as part of the instructional team, foster civic 
responsibility, and make students active participants in their learning. The Hands-On 
Science and Technology Center (HOST) teaches science principles to elementary, 
junior high, high school and adult audiences. Students connect theory with solving real 
physics and science problems. The curriculum connects science with engineers to work 
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with students. The Kem County Partnership applies the HOST model in the Museum 
Project, which trains students as museum tour guides. Students are involved in every 
aspect of the museum project planning and implementation. 

• The Workforce Silicon Valley LP worked with the Multimedia Learning Collaborative 
to refine a web site multimedia project for work-based learning. The project-based 
assignments are evaluated by industry partners, providing direct feedback to the 
classrooms. 



Support for stakeholder involvement 

Creating and maintaining a high level of stakeholder involvement is one of the greatest 
challenges facing LPs. Employers, labor organizations, and public and private agencies must 
understand the value of the STC enterprise and encounter no more than minimal obstacles to 
participation. Even among organizations and agencies that endorse the concept of STC, 
participation hinges on excellent and ongoing communication with school personnel. In schools, 
as well, teachers, counselors, and other staff members need information about how STC activities 
will directly affect students. And they require professional development to carry out their 
supporting roles. 



Across the state, LPs are engaged in a wide variety of connecting activities aimed at creating and 
supporting high levels of school and community participation in the STC effort. For the most 
part, these involve establishing liaisons between schools and employers, providing professional 
development for school staff, and linking all of the key stakeholders to collaboratively plan 
and/or develop curricula. 

• Throughout Orange County there are various STC-related efforts such as Tech Prep, 
four ROC/Ps, approximately 80 high schools with independent programs, 60 chapters 
of Future Scientists and Engineers, about 70 schools partnering with Junior 
Achievement, and over 1,700 local school-business LPs. Between 500 and 600 notices 
are sent out monthly to notify all stakeholders of Orange County Coalition/Vision 
20/20 meetings. The emphasis of these meetings is collaboration among all 
stakeholders and between schools and businesses. Programs representing business, 
education, and community-based organizations are scheduled so that all participants 
become familiar with STC resources. Also, the meetings are held on a geographical 
rotational basis to gamer as large and diverse an attendance as possible. 

• Business Liaison consultants in Stanislaus County have continued to increase the 
number of effective school and business LPs participating in STC. Two consultants 
have been hired and oriented in Tuolumne County and are meeting regularly with 
educators. They also meet with local business/industry people to explain the role of 
business in STC, and to initiate new school and business LPs (Yosemite Community 
College District STC). 
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• The UNITE-LA LP has adopted the following philosophy: We always assume any plan, 
action, or idea we want to accomplish is already being done well by someone and/or 
some organization and therefore our first step is always to find . . . them and attempt to 
connect them to the “system”. For example, UNITE-LA has been successful in 
attracting IBM to provide systems consulting expertise in developing a clearinghouse 
strategy. IBM has also contributed monetary donations and two personal computers. 



Access For All Students 

Because the STC effort aims to engage all students, LPs and schools face special challenges 
related to ensuring access for all groups. With the help of their community partners, educators are 
making special efforts to engage students who might otherwise be left out of learning 
experiences in the workplace or integrated career and college guidance activities. These include 
at-risk students, English Language Learners, and students with disabilities. LPs attempt to 
include all students by building on programs that already target specific groups, such as Tech 
Prep, and Adult and Alternative Education. 

• Attention to “All Students” continues to be a focus for the Partnership for Tomorrow in 
San Joaquin County as they forge new avenues of assistance with a diverse population. 
There has been continued progress in the Alternative and Special Education setting with 
a Guidance Counseling Manual being completed by the Counseling Task Force. Other 
program possibilities are being explored to help individualize career goals for students 
in Alternative Education. A strong alliance has been developed with Special 
Education's Job Development/Job Coach Program, and a job shadowing program will 
involve this population. A very positive interaction has begun with the Southeast Asian 
community through the Lao KJmu Association. A creative plan for placing an intern, 
sponsored by the Lao Khmu Association, is being developed. The intern will work in 
the Partnership for Tomorrow office with responsibilities to do outreach in the 
Southeast Asian community. 

• The North Coast STC Consortium has designed a comprehensive lab for all students. 
The lab is available for distance learning, access to a One Stop Center, and Internet 
access for high school and adult school students. The lab also targets special groups by 
including other at-risk populations. These include CalWORJCS recipients, the EDD 
unemployed, minorities, pregnant and parenting teens, and post-secondary students. 
Computer literacy and training are provided for staff and students. 

• The Long Beach STC Consortium has implemented systemic changes that are aimed at 
ensuring long term commitment to providing access for all students. They have 
combined the Youth Council branch of the PIC and the STC Governance Committee to 
form one central body: The Youth Governance Committee. In addition, the Long 
Beach STC Consortium attempts to include parent and student groups, special 
education representatives, probation representatives, the Neighborhood Resource 
Center, and the Chamber of Commerce. 

• San Francisco STC actively identifies and recruits community-based organizations and 
programs that can provide resources for economically disadvantaged, limited English 
speaking, disabled, academically disadvantaged, and other at-risk youth. One of the key 
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resources SFSTC seeks in recruiting these partners is their experience in reaching, 
enrolling, and serving their constituencies. They have worked with Jobs for American 
Graduates (JAG) and collaborated in the preparation of the Youth Opportunities Grant 
(Workforce Investment Act) with Jobs for Youth. 

• Due to the agricultural base of Riverside County LP desert communities, a large 
migrant population exists in the Eastern portion of Riverside County. The Riverside 
County Office of Education (RCOE) serves this population through the Mirant 
Education, Even Start, and Pace programs. Thirty students were offered individual 
tutoring, mentoring, and career counseling. Five students received the services of a 
bilingual teacher, which allowed two of the students to return to continuation school 
and receive their diplomas. Three students received GED preparation, and a bilingual 
teacher worked with the remaining students personally. All but two students graduated 
that June. 



Summary of Qualitative Findings 

Analyses of LPs’ Performance Matrices, quarterly reports, and evaluation reports show that the 
state gave LPs considerable latitude in designing STC activities and strategies. Although it is 
difficult to discuss progress toward reaching specific goals based on these documents, or to draw 
inferences about the STC activity levels within or across LPs — including the number of schools, 
students, teachers, and businesses involved or the number of hours devoted to the activities — 
these data provide a rich, qualitative description of LPs’ efforts related to four areas of STC: 
school-based learning, connecting activities, work-based learning, and access for all students. 
These data also informed the design of Phase II of the CA STC evaluation. 



School-based activities 

Data suggest that LPs are using a wide variety of school-based activities and strategies, 
including: 

• career exploration and college awareness activities, which typically begin in the 
elementary grades and continue through high school. Activities range in length, 
intensity, breadth, and depth. Some LPs try to reach a high percentage of students at all 
grade levels by offering low-intensity activities such as a career fair, while others 
choose to concentrate their efforts on specific grade levels and offer more in-depth 
activities to fewer students; 

• career counseling and guidance, where counselors provide guidance on career choice 
and opportunity rather than focusing almost exclusively on class scheduling; 

• developing or revising curricula based on labor market information and employer input; 

• teaching employability skills in the classroom, including SCANS-based skills that were 
developed with a board of business and industry representatives. A number of LPs are 
promoting the emphasis of SCANS skills by providing teacher training and the 
development of SCANS-based curricula; 
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• teaching technical skills in the classroom through the incorporation of industry- 
recognized technical standards into the curricula; and 

• informing students, employers, and teachers about legal and safety requirements for the 
work environment. While some LPs post legal requirements, others offer in-depth 
training for teachers and employers. 

Connecting activities 

Available data about connecting activities conducted by LPs were limited. Data suggest that, as 
was the case with school-based learning, LPs are using a variety of different approaches to link 
work-based and school-based activities, including: 

• creating direct links between employers and teachers by hiring a teacher-employer 
liaison or offering teachers opportunities to spend time in the workplace (teacher 
intemships/extemships); and 

• offering business leaders the opportunity to work directly in schools through programs 
such as “Principal for a Day” or by helping with curriculum development. 

Work-based learning 

LPs often connect work-based learning opportunities to other local programs such as Tech Prep 
and ROC/P. Overall, qualitative data suggest that LPs are using a variety of approaches to help 
students learn about the real-life demands of the workplace. 

As would be expected, the following work-based activities are common: 

• internships, 

• job shadowing, and 

• community service. 

To facilitate participation and support for work-based learning among employers, labor 
organizations, public and private agencies, teachers, counselors, and other staff members, LPs 
are: 

• providing professional development for stakeholders; and 

• hiring personnel to serve as liaisons between employers and schools. 
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Access for all students 

In their efforts to provide access to STC for all students, LPs commonly build on programs that 
already target specific groups that serve a diverse group of students. Quarterly reports indicate 
that LPs are: 

• coordinating efforts with programs such as Tech Prep and Jobs for Youth to recruit 
community partners and to find placements for work-based learning opportunities; and 

• identifying and recruiting community-based organizations and programs that can 
provide resources for at-risk students, such as Jobs for American Graduates (JAG) and 
Jobs for Youth. 



Stakeholder Participation: Early Quantitative Findings 

The following is a discussion of the most recent, comparable quantitative data available on major 
STC implementation activities in California LPs and draws comparisons to national figures, 
where appropriate. It is based on data from the STW Local Partnership Survey (LPS) conducted 
in fall 1998 by Mathematica Policy Research, as reported in The 1998 National Survey of Local 
STW Partnerships: Data Summary. This discussion follows the general structure of the 1998 
Mathematica LPS report, and readers are directed to Appendix E for a complete summary of data 
from the 1998 LPS.'* 



In the third of four national mail surveys, LPs across the country were asked about their 
organizational structure, implementation activities that were ongoing during the 1998-99 school 
year and, in some cases, about activities that were completed in 1997-98. Twenty-three of 
California’s then 28 LPs responded to the survey, yielding a response rate of 82%. 

Partnership Participants 

The STWOA recommends that a wide array of education and other groups cooperate to meet 
community needs by establishing effective LPs. These groups typically fall into four broad 
categories: education institutions, training institutions, business and labor organizations, and 
other entities, such as workforce development boards, government agencies, and community- 
based organizations. To create a LP in California, the following partners are required: employers; 
LEA Representatives (elementary, middle, and secondary); post secondary representatives; local 



Lara Hulsey, Michelle Van Noy, and Marsha Silverberg, Mathematica Policy Research, Inc., 1998. 
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educators; organized labor; students; and parent organizations. Here is key information about LP 
participants. 

• Education institutions most frequently act as lead organizations for California LPs, in 
1998 representing 47.8% of the total. This is somewhat lower than the 56% of LPs 
nationwide that were led by an education agency or institution. 

• A large number of California LPs (30.4%) were unwilling to designate a single lead 
organization when responding to the survey. This percentage is slightly higher than the 
25% figure nationwide and suggests that in California there is a high level of 
organizational collaboration in partnership affairs. 

• There is a very even distribution among high schools, middle schools, and elementary 
schools in their representation within California LPs. In 1998, there were 365 high 
schools in the 23 reporting LPs (an average of 15.9 per partnership), 338 middle 
schools (average = 14.7) and 310 elementary schools (average = 13.5). 

• In the business and labor category, private sector firms had the largest representation in 
California’s LPs by a very wide margin. In the 23 LPs that responded to the survey, 
membership included 1,172 private sector firms, 41 business/industry or trade 
associations, 56 Chambers of Commerce, and 47 labor unions. (However, organized 
labor was also represented by 22 registered apprenticeship agencies that were included 
in a separate category with 50 other training institutions.) 

• Every California LP that responded to the LPS included at least one post-secondary 
institution, and many had more than one. On average across the 28 reporting LPs, there 
were 1.8 two-year post-secondary institutions and 1.5 four-year institutions. Given the 
fact that there are more two-year post secondary institutions than four-year institutions 
it would be expected that there would be a greater number of two-year institutions. 
These data suggest that relative to high schools, participation from two-year post 
secondary institutions has been less than representative. 



Career Awareness and Development 

The STWOA builds on other federal education initiatives by promoting career counseling and 
guidance to help students plan for their future education and career success. It establishes a 
progression of activities designed to help students meet their goals. The sequence begins with 
career awareness in the early grades and continues with career exploration and preparation as 
students move through middle and high school and into post-secondary education. In California 
and nationally, career awareness and development activities have been the leading edge of the 
growing STW movement. 

• In 100% of California secondary schools, post-secondary institutions, and alternative 
education providers that reported on their career development activities in the LPS, 
educators were offering these services to students. (However, only 73.6% of secondary 
schools, 52% of middle schools, and 33.3% of alternative education providers reported 
this information on the survey. Schools that left this part of the survey blank were 
probably less likely to offer career development activities to students.) 
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• At the high school level, career interest inventories (54.3%), individual career 
counseling (52.8%), and activities integrated into academic and vocational classes 
(51.4%) were the most frequently offered components of a career development model. 
However, many schools also gave students scheduled access to career centers (47.4%) 
or to separate career awareness classes (39.8%); arranged career fairs (41.2%); or 
helped students develop individual career and educational plans (44.3%). 

• Roughly one-third of post-secondary institutions in LPs offered the same array of career 
development activities as did secondary schools, with the exception of written 
individual student plans, which are less relevant for post-secondary students. 



Promoting Student Access 

Both the federal STWOA and California’s plan for STC implementation stress the need for 
systemic change. STW is intended to be more than another targeted program that reaches only 
certain segments of the student population. The goal is for all students to receive broad exposure 
to the key elements of an STC approach: school-based learning, work-based learning, and 
connecting activities. To achieve wide student access requires that LPs and schools develop and 
implement program elements that are specifically designed to attract and succeed with hard-to- 
serve groups. 

The LPS identifies eight special populations to assess the extent to which LPs and their member 
schools are implementing strategies to include diverse groups of students. These groups are 
minority students, English Language Learners, students with disabilities, students who are 
economically or educationally disadvantaged, pregnant or parenting students, out-of-school 
youth, students pursuing occupations that are nontraditional for their gender, and academically 
gifted and talented students. Like LPs nationwide, in California there is evidence that LPs are 
working to make STW activities available to a broad array of students by including them in 
decision-making and offering targeted services and programs. Although conclusive data are not 
available about the extent to which these special population participate in STC efforts, results of 
the LPS suggest the following: 

• Depending on the particular student group, between one-quarter and one-third of 
California LPs require that representatives of these students serve on their governing 
boards. 

• About one-half to two-thirds of California LPs offer funds for special activities related 
to the needs of these groups, depending on the particular student group. 

• Between one-quarter and one-half of California LPs train members of the business 
community to work with members of these student groups. 




70 



52 



III. Efforts To Create a School-To-Career System in California 



• At the level of individual schools, the most popular methods of increasing access by all 
students are using targeted promotional materials (about 80% or more of schools, 
depending on the student group) and special career guidance (also about 80% or more 
of schools). 

Career Majors 

One of the unique elements of the STWOA is a focus on career majors — a strategy for 
organizing the high school curriculum around broad career areas or occupational clusters. In 
many respects, this emphasis on career-focused education is an outgrowth of earlier education 
reforms, such as the California Partnership Academies, that attempt to make education more 
relevant for students. In California and elsewhere, career majors take many shapes and forms. At 
one end of the spectrum are situations where counselors ask students to identify a career they 
want to plan for and to use that career goal as one element in selecting courses, particularly 
electives. At the other end of the spectrum are academies and magnet schools, where students, 
teachers, and courses are all organized around various career themes. 



The LPS has started to identify the various definitions that LPs have in mind when they establish 
career majors, to determine the prevalence of these various types of career majors, and to show 
the prevalence of particular industry groupings within career major structures. 

• In California, 100% of LPs responding to the survey include at least one school that 
offers some type of career major to high school students. This compares to only 80% of 
LPs nationwide. 

• At the level of individual schools, however, only ,52.3% of California high schools offer 
some type of career major, which is roughly equivalent to the percentage that do so 
across the country. 

• Written course sequences that help students select their courses are the most common 
form of career majors in California.’^ Almost 96% of LPs have at least one school that 
has created these sequences, and they are available in 47.2% of high schools in the 23 
California LPs. This compares to 76.9% of LPs and 46.6% of high schools nationwide. 

• Less common both nationally and in California are schools that group some (36.1%) or 
all (2.8%) students by their area of career interest. Nearly 60% of California high 
schools in the 23 LPs do not cluster any students around a career major area, a 
percentage that is almost identical to the national figure. 

• Like other parts of the nation, in California the two most frequent industry clusters for 
career majors are business/marketing/finance and engineering/industrial technology. 
However, the percentages of California schools offering these career majors are 
somewhat lower than the figures are nationally: 28.7% versus about 40% for 



Written Course Sequences are required courses to complete a career major, established to help students make 
course selections, and are the most common form of career-focused education in California. 
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business/marketing finance and 27.3% versus 33% for engineering/industrial 
technology. 

• Despite the relatively high percentages of California LPs that offer some type of career 
major program in at least one high school, the actual numbers of students who select a 
career major is small. As a percentage of 12"' graders, slightly less that 6% had selected 
a career major by the time they were in their final year of high school. This was about 
half the national figure of 11.7%. In addition, only 42.3% of California high schools 
responding to the LPS were able to report on the number of students selecting career 
majors. This could suggest that the actual representation of students with career majors 
is even less than 6%. Alternatively, the discrepancy between the percentage of students 
in California who reportedly have a career major (6%) and the national figure (11.7%) 
could suggest fragmented data collection and reporting systems. 



Curriculum Integration 

Similar to career majors, the concept of curriculum integration is also open to a wide variety of 
inte^retations. Over more than a decade, school administrators, curriculum specialists, teachers, 
and education strategists have defined and implemented integration in many ways. They have 
changed course content within individual classrooms; made structural changes in whole schools 
(like career academies, career clusters, or magnet schools); introduced new instructional or 
pedagogical approaches (like integrated projects or group work); and connected classroom work 
to activities in the workplace. 

• In both California (56% of secondary schools) and the nation (60.1%), the most 
common approach to integration is developing applied units or projects within the 
school. 

• In California high schools, there is a rapid drop-off when it comes to the prevalence of 
other integration strategies. Only about one-quarter to one-third of high schools use 
approaches such as teaching all aspects of various industries, purchasing commercial 
applied academics packages, using state-developed applied curriculum materials, 
academic- vocational team teaching, or teacher collaboration in groups to develop 
career-related curriculum units. At the national level, the prevalence of several of these 
integration practices ranges between 40% and almost 50% of high schools. 

• Schools can also pursue curriculum integration through collaborations involving 
secondary schools, post-secondary faculty, and employer or labor representatives. In 
California, 29.3% of high schools indicated that their teachers collaborate with college 
faculty on new or revised curricula, while that figure is only 26% nationally. On the 
other hand, only 23.9% of California high schools collaborate with employers on 
curricula for classrooms or work sites, while 32.5% do so nationally. 

Secondary/Post-Secondary Partnerships 

Helping students identify their career goals, understand the importance of education or training 
after high school, and prepare for further education are some of the most important goals of STW 




72 



54 



III. Efforts to Create A School-to-Career System In California 



efforts. One of the key strategies for schools to achieve these goals is for secondary schools and 
post- secondary institutions to develop strong connections. Information sharing, various types of 
collaboration around curricula and standards, and administrative structures that facilitate the 
transition from high school to post-secondary education are some of the most frequent types of 
connections between secondary and post-secondary schools. 

• Overall, the strategies that California uses to link high schools and post- secondary 
institutions (e.g., articulation agreements, collaboration around standards) are similar to 
the rest of the nation. 

• Sharing employer networks and contacts (48.9% of high schools in California and 
46.9% across the country) is the most common type of collaboration. This is closely 
followed by sharing labor market information (48.6% and 47.3%, respectively). 

• In terms of administrative structures, agreements allowing students to have dual 
enrollment at their high school and a college, as well as articulation agreements 
granting advanced college standing for high school work, are very popular in California 
and elsewhere. Close to half of California high schools (48%) permit dual enrollment, 
and almost 60% have secondary/post- secondary articulation agreements between 
secondary and post-secondary schools that grant advanced standing. 

• The least frequently used strategy for helping students advance to college work is 
granting college credit for high school work-based learning. Only 13.3% of California 
post-secondary institutions and 19.7% of institutions nationwide grant this type of 
credit. 



Work-Based Learning 

Work-based learning is one of the cornerstones of the STWOA and California’s STC initiative. It 
is a mechanism for helping students see the connection between classroom learning and future 
careers and a vehicle for helping them learn about career alternatives. There is wide variation in 
the types of activities that schools have developed to give students exposure to the workplace. 
These range from short-term exposure to various jobs and industries — ^through work site tours or 
brief job-shadows — ^to intensive activities that give students a more thorough understanding of a 
career area and may actually help them develop some job skills. Long-term internships and paid 
employment that is connected to a career major are examples of these intensive activities. 



• Among individual LP high schools, only work site visits and job-shadows have 
captured widespread attention as part of the curriculum. Almost 43% of high schools 
offer these low-intensity opportunities, while percentages for other activities range from 
9.9% (assignment to a workplace mentor) to 31.8% (unpaid internships during the 
school year). Nationally, it is also the case that low-intensity activities are most 
prevalent. 
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• The LPS identified four types of work-based learning that can be connected to career 
majors: paid school-year jobs, unpaid internships or jobs during the school year, paid 
summer jobs, and unpaid summer internships or jobs. For all of these, California has a 
much higher percentage of LPs where at least one high school offers the activity. 
However, while more California high schools offer unpaid learning opportunities and 
paid summer jobs than schools elsewhere, fewer provide paid jobs during the school 
year. 



Summary of Quantitative Findings 

Survey data provide a picture of STC during the early stages of California’s implementation of 
STWOA, and allow some comparisons to data at the national level. Major findings fi'om the LPS 
are summarized below. 

• Almost one-half (47.8%) of LPs are led by education institutions, but data suggest that 
almost one-third (30.4%) are led by more than one type of organization, indicating a 
high level of collaboration between different groups within LPs. 

• There is a fairly even distribution of high schools, middle schools, and elementary 
schools in LPs, and every responding LP included at least one post-secondary 
institution. 

• Private sector firms have a much higher level of participation in LPs than trade 
organizations, labor unions, or Chambers of Commerce. 

• With regard to career awareness and development, career inventories, individual career 
counseling, and activities integrated into academic and vocational classes were the most 
fi"equently reported STC components at the high school level. Approximately one-third 
of post-secondary institutions also offered these activities. 

• LPs are promoting access to all students by using a variety of strategies, including: 
requiring that representatives of targeted groups (e.g., English Language Learners; 
students with disabilities) serve on their governing boards; offering fonds for special 
activities related to the needs of these groups; training individuals fi'om the business 
community; and using targeted promotional materials and special career guidance. 

• LPs are using a variety of strategies to focus high school students on career majors. The 
most common approach to career majors in California involves providing students with 
written course sequences to help them make course selections. Far fewer schools group 
students by their area of career interest. Data indicate that less than 6% of high school 
seniors in California LPs had selected a career major. 

• Collaboration between secondary and post- secondary institutions most commonly 
involves sharing employer networks and contacts (48.9%) or sharing labor market 
information (48.6%). Moreover, close to half of California high schools allow students 
dual enrollment in high school and college. 

• With regard to work-based learning, the most prevalent activities in high schools in 
California LPs include work site visits and job shadowing (about 43%). High intensity 
activities such as workplace mentoring and unpaid internships are offered far less 
fi'equently (in approximately 10% and 32% of high schools, respectively). 




74 



56 



III. Efforts To Create a School-To-Career System In California 



Local Partnership Sustainability Efforts 

As LPs reach the final stages of federal funding, STC related strategies become a much greater 
concern. Many sustainability efforts defined by LPs are closely connected, or even identical, to 
the STC implementation activities these same LPs report. In their reports to the state, LP leaders 
group sustainability efforts in the following ways: 

• recruiting partners and local business; 

• connections with community-based organizations; 

• linkages to post-secondary institutions; 

• communication strategies to gain support from various stakeholders; 

• regional media collaboratives; 

• professional development of stakeholders; and 

• capacity building. 

Pursuit of these sustainability activities does not assure a sustainable STC system. However, they 
are recognized as strategies that help LPs individually and collectively support sustainability. 

Recruiting Partners and Local Business 

The sustainability of an STC system depends heavily on the recruitment of representatives from 
key stakeholder groups and, more importantly, on effective collaboration among them. These 
stakeholders should include local businesses, community-based organizations, post-secondary 
institutions, and parents. Nurturing these relationships takes extensive time and communication 
so that all see the rewards and positive outcomes of STC both for themselves and for the 
students. 

Significant involvement from local employers is especially essential for many aspects of the STC 
effort, especially for curriculum plaiming, work-based learning activities, and career guidance. 
Since local businesses are in the prime position to relate the expectations and demands of the 
workplace to educators and to schools. In recruiting businesses, LPs need to encourage long-term 
commitments from local businesses as opposed to participation in one-time activities. 

Many LPs discussed the opportunities businesses provide for internships, job-shadows and other 
work related activities. However, few LPs reported long-term resource and/or financial 
commitments from businesses. For this reason, it is unclear to what extent work-based learning 
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opportunities will be available to students after the federal funds to the state are fully expended. 
Providing the resources, in terms of both staffing and finance, for continually active contact with 
the industry will be a challenge for all LPs. However, sustainability will only be realized through 
a focus on long-term commitments, not short-term buy-in. 

• San Francisco STC has made efforts to secure resource commitments from employers 
and translate general support agreements into work-based learning opportunities in 
pathway occupational areas. Between 500 and 700 specific employer commitments 
were made in 1999, and about 300 work-based learning opportunities were generated to 
support the LP’s summer internship program. 

• The Orange County CoalitionA^ision 20/20 specifically recruited the Executive 
Director of the Orange County Central Labor Union to serve on its Board of Directors. 

• The Monterey Bay Regional LP has involved various agencies and business in 
implementing and promoting STC initiatives. Business and industry partners include 
the Business/Education Alliance of Monterey Bay (BEAM), the Business/Education 
Student Transitions (BEST) and the Your Future is Our Business collaborative. Your 
Future is Our Business maintains a database of over 600 businesses that agree to 
provide various levels of work-based learning opportunities to students and staff; BEST 
maintains a similar database of over 400 employers. This coordination will be 
beneficial in sustainability if the Monterey LP can provide continued access and 
encourage students to use these available databases. 
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Connections with Community-Based Organizations 

Community-based organizations can offer a strong support system for STC concepts and 
practices within a community. At the same time, STC has the unique opportunity of bringing 
together community-based organizations with schools and other community partners who share 
their interest in youth development. For example, collaboration between Chambers of Commerce 
and County Education Offices helps students find quality internship opportunities and introduces 
them to careers and employment opportunities in their local areas. Among other important 
organizations included in these collaborations are Private Industry Councils, Offices of the 
Employment Development Department, Parent Teacher Associations, Minority Councils, and 
Workforce Development Councils. 

• At California Works! the Working for Interagency Solutions (WINS, a six-county 
partnership) Committee held a summit. Attendees included several community-based 
organizations, such as the Yuba-Sutter Head Start Program, Girl Scouts, Boy Scouts of 
America, Sutter County Library, CTEC (One-Stop), Private Industry Council, Alliance 
for Hispanic Advancement, Department of Rehabilitation, Migrant Education, 
California Human Development Corp., and Friday Night Live (part of California 
Works!). The meeting not only provided attendees with information about STC, but the 
attendees had the opportunity to share information about their services with each other 
and discuss ways in which interested agencies can work together by sharing 
information, ideas, and resources. The director of the Tri-County ROP made a 
presentation addressing the role of ROP in STC; a representative from Friday Night 
Live discussed the critical need to incorporate mentors into the lives of students; the 
Freemont-Rideout Health Group also made a presentation highlighting hiring trends in 
the health industry; and a panel with representatives from various personnel agencies 
discussed the skills employers within the community are seeking in potential 
employees. 

• The Ventura County STC Network is working with the community and GTE to provide 
STC opportunities. The GTE Community Connection is a program that targets soon-to- 
be emancipated foster children to help place them into appropriate internships and 
assign them a mentor within GTE. In another joint effort, Ventura County STC 
Network is producing a video documentary to feature the Fillmore High School 
Construction Career Pathway and its linkage to the Habitat for Humanity project in 
Pirn, California. The production of this video features linkages with the Ventura Adult 
Education Technology Development Center, Columbia Broadcasting System (CBS), 
Ventura County STC Network, Association of General Contractors of California, 
Fillmore Unified School District, and Habitat for Humanity. 

• The El Dorado County JOB ONE LP has formed strong working relations with several 
Chambers of Commerce. These include three Western Slope Chambers of Commerce 
and the County Chamber of Commerce. The El Dorado Chamber of Commerce 
committed to “Expo 2000,” combining a job fair and business exposition. By aligning 
with several community-based organizations, the El Dorado LP is able to expand their 
exposure and increase support for STC activities. 



ERIC 
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Linkages to Post-Secondary Institutions 

Creating strong linkages with post-secondary institutions (PSIs) is critical for promoting the 
value of higher education and academic skills and maintaining pathways to higher education for 
students participating in STC. The types of resources available to PSIs, and the important role 
they play in the local community as an institution, make them key partners in efforts to sustain 
STC. Post-secondary institutions offer a rich variety of benefits to STC. In the most immediate 
sense, linkages with PSIs ensure the continuation of students’ pathways, guarantee recognition of 
students’ acquired skills and certificates, and enrich career and education awareness activities 
through their participation. 



LPs are pursuing two main strategies for making connections with post-secondary institutions: 
building on linkages established by existing vocational and academic programs, such as Tech 
Prep and ROP, and creating new articulation agreements between LP schools and PSIs. 

• The Santa Barbara High School (SBHS) ROP automotive and business courses are 
articulated with Santa Barbara City College (SBCC) classes, and students can receive 
concurrent enrollment credits. Students are eligible to take the SBCC on-line Careers 
Exploration Class, Personal Development 191 A, as well as register for concurrent 
SBCC credit when taking the Work Experience Class or Advanced Placement English 
at SBHS (Santa Barbara STC Consortium). 

• The Paramount Unified School District in the Los Angeles Youth Development 
Partnership has developed strong links with Cerritos College. A course outline has been 
developed for a networking class — ^basic courses will be taught at the high school with 
advanced courses being taught at Cerritos. Students may elect high school or college 
credit; completion of the program leads to certification. 

• Temple City Unified School District, also in the Youth Development Partnership, is 
part of a new Tech Prep grant with Pasadena College that focuses on Computer 
Information Systems. In addition, all tenth grade students and their families were 
invited for private counseling sessions to identify career and future plans after 
graduation and to develop a four-year plan for reaching their goals. 
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Communication Strategies to Gain Support From Various Stakeholders 
LPs are employing several tools to communicate the success of STC and to gain support from 
stakeholders. A wide variety of activities occur here, from informational newsletters and videos 
to advertising, to including stakeholder collaboration in communication products. Special efforts 
are made to target adult, special education and alternative education schools. The challenge will 
be for LPs to continue to utilize these tools to maintain the interest generated for STC, maintain 
and increase school-based and work-based learning activities, and gamer external contributions 
once federal funds end. 

• In the Long Beach STC Consortium, a project involving the Port of Long Beach was 
taped by CBS and aired in June of 1999. This type of involvement creates positive 
exposure for LPs and businesses. 

• Within the Santa Barbara STC Consortium, the Santa Ynez STC agency commissioned 
an STC awareness video. This video underwent extensive review by an Ad Hoc 
committee and the Advisory Council. Advisory Council members will use the 
informational video to inform and recruit the local community (parents, businesses, or 
local organizations) as partners in the Santa Ynez STC agency. 

• At California Works! the general brochure and Work-Ready Certificate were translated 
into Spanish and Punjabi by the Migrant Education office. The translations were an in- 
kind contribution to the LP. The translations will be formatted and distributed to 
students and parents within the community. In addition, a presentation was developed 
that focuses on providing employers with information on STC and recruiting business 
participation. 

• The Monterey Bay Regional Partnership is using positive evaluative results to promote 
STC initiatives. The LP has developed and implemented a longitudinal study 
encompassing all 11th and 12th graders in the two county area. The information has 
been disseminated to individual schools and to the business community through their 
Project Board. They are moving into Phase Two of a four-year study. Much of the 
information gathered confirms that students are eager for more career information and 
training, and see STC activities as encouraging them to continue their education. 

• Some LPs are targeting all students by providing information in different languages and 
to specifically targeted groups. The Verdugo STC Coalition has completed brochures in 
English, Spanish, Armenian, and Korean. The coalition have also attended five separate 
Welfare to Work meetings to include this population. 
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Regional Media Collaboratives 

Regional Media Collaboratives are able to work with a variety of districts to promote best 
practices. One of the main Regional Collaboratives that has a strong media component is 
BaySCAN. BaySCAN is able to promote STC activities for the LPs through relationships with 
KPIX Channel 5. In addition, the promotion of activities such as those found at Charles Schwab, 
provide high profile exposure. 



Professional Development of Stakeholders 

Effective professional development for all stakeholders in the STC system is essential for the 
long-term survival of these school improvement efforts. As California LPs are looking to the 
future, they are investing in professional development activities using a wide variety of 
approaches. 

• In 1999, the Butte County STC Consortium hosted a “School to Careers Summer 
Internship” for educators. Twenty-six businesses hosted eighteen educators 
representing the entire county and grade levels one through fourteen. “The workshops 
provided teachers with SCANS, NCDG (National Career Development Guide), work- 
based and school-based STC activities, curriculum integration training, and business 
and student guest speakers. Interns were also required to document daily examples of 
SCANS being demonstrated at their work site, conduct a business informational 
interview, write an internship summary, prepare a lesson plan utilizing their internship 
training and expertise, and identify a mini-grant focus for their school.” 

• Eight staff development workshops were offered by the East San Gabriel Valley LP, 
covering a range of implementation issues and concepts. Topics included integrating 
curriculum, guidance and counseling strategies, non- traditional occupations^ “how to” 
products and guidebooks, portfolio assessment, articulation, comprehensive reform, and 
job shadowing. 

• During the summer of 1999, teachers were given the opportunity to job-shadow a 
professional in a business outside of the education field at the Partnership for 
Tomorrow. The weeklong experience was arranged by a Partnership for Tomorrow 
liaison working with community businesses that are partners in the STC program. The 
various business participants included law enforcement agencies, hospitals, newspapers, 
and agri-business. 

• The Solano County STC program has established a unique professional development 
program for its teachers. They have developed a creative “Teachers in the Workplace” 
training course whereby county teachers are provided release time to participate in 
hands-on, intem-like work experiences. Local companies such as Clorox and ALZA 
Corporation developed the curriculum and job training focus. In addition, these 
companies paid each teacher a salary during their internship and developed a detailed 
exit interview process to capture lessons learned for future work opportunities. 

• The Solano County STC program also offered professional development in the form of 
“Principal for a Day.” This program gives the business community the opportunity to 
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experience the school setting and facilities, as well as participate in a dialogue 
regarding the development of curriculum influenced by the realities of the business and 
industry environment. Over 50 business leaders have participated. 



Capacity Building 

In order to implement, support, and sustain STC efforts, LPs are involved in the following 
categories of activities. 



Construct sustainable databases 

Several LPs are taking advantage of technology to promote their STC initiatives and to provide 
timely and relevant data. Some LPs have installed new computers and software for STC 
initiatives, while others build on existing databases such as those used for Tech Prep or the EDD. 
Databases are made available with directories of employers and available work-based positions. 
In some cases the databases are a direct listing of employers offering job or internship 
opportunities. In other cases, the databases are interactive tools with links to several other sites 
such as the COC and local community colleges and universities. These provide informational 
links with K-16 for students, teachers, employers, and administrators. Although implementing 
databases is an effective analytical tool, maintaining these databases after federal funding 
terminates will be a major concern for sustainability. 

• The Riverside County Economic Development Agency (EDA) has made available to 
the public an integrated database for employment, training, and work-based learning 
sites via One-Stop’s kiosks placed countywide. In addition, their ERISS web site link 
provides access from any Internet-ready computer terminal to a database of over 12,500 
businesses in the Inland Empire. The database includes a wealth of information 
resources including up-to-date job lists, employer profiles, hiring practices, and 
education requirements for position openings. To date, over 15 kiosks have been placed 
in easily accessible locations throughout Riverside County. 

• Linking Education and Economic Development (LEED, Sacramento) is investing in 
equipment and software to increase the efficiency and effectiveness of the LEED 
Information Network Collaborative (LINC). LINC serves as the regional connecting 
activities system and is used to facilitate the work of Industry Education coordinators. 

• San Luis Obispo Chamber of Commerce and the Education to Career Connection 
collaborated on an Internet electronic database for businesses to provide work-based 
learning opportunities to educators and students. The web site is accessible from the 
Chamber Home Page. The LP selected New Image Technologies from among bidders 
to develop the “education2career.com” web site. This site will be accessible from the 
San Luis Obispo County Office of Education’s home page and will link to the Chamber 
site. 

• Vision 20/20 has made efforts to make curriculum information available across the 
county electronically. The Vision 20/20 web site will provide electronic access to 
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instructional materials produced by educators who participated in paid summer 
intemship/material development opportunities. Over 200 requests for materials had 
been processed at the start of 1999. 



Link to other programs 

Another effective means of sustaining STC is building alliances with other programs currently 
implementing similar concepts. Programs such as the Regional Occupation Program (ROP), 
career pathways, and career clusters already incorporate curricula that offer applied learning in 
career majors or a general career field. WIA and Tech Prep programs offer a significant base for 
including at-risk populations, for supporting secondary to post-secondary linkages, and for 
apprenticeship funding programs. LPs may draw from WIA and Perkins III funding to continue 
implementing STC activities and can also partner with Private Industry Councils (PIC) that were 
originally funded under JTPA. These councils provide insight on local industry and have been 
active in collaboration of STC initiatives. Also, coordination with Tech Prep facilitates the 
creation of career majors in schools, and Tech Prep programs have already established 
recognized skills certificates. 

• EASTBAY Learns has partnered with several community organizations for program 
support and alternative funding. The Contra Costa County Office of Education has 
committed to a full time STC coordinator. Several school districts took advantage of 
funding for after school and mentoring programs. A Health Careers Learning 
Collaborative will be implemented through foundation funding. JTPA funds have been 
secured for STW related activities. EASTBAY Learns also capitalized on Regional 
Workforce Preparation and Economic Development Act funds for various STC 
programs. 

• The San Francisco LP has helped to integrate two programs from its school district that 
serve at-risk and low-income youth. The Summer Training and Education Program 
(STEP) and the Summer Youth Employment and Training Program (SYETP) both have 
experience with developing and implementing STC concepts. They have strong 
employer connections and support services that increase at-risk students’ opportunities 
to participate in work-based learning activities. 

• The collaboration between Lynwood Unified School District and WIA/Hub Cities 
Consortium resulted in placing approximately 250 middle and high school students in 
summer employment within the district/community of the Greater Los Angeles County 
LP. In addition, the Youth Corps of the Southeast Lost Angeles County Private Industry 
Council (SELACO PIC) offers opportunities to disenfranchised youth. Youth are 
exposed to many career options and training opportunities for meaningful and gainful 
employment. The Youth Corps has a fully operating job club equipped with computers 
with Internet access, job search resources, daily newspapers, and other resources that 
facilitate effective job search activities. 
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Accountability /evaluation 

The concept of accountability/evaluation differs slightly from other activities to sustain STC 
systems. Although accountability/evaluation is not a direct means of sustaining STC, it forms an 
essential base for sustainability through research in two ways. LPs can use research to evaluate 
the effectiveness of their implementation strategies. 



Second, assuming favorable evaluations finding, LPs can use those findings to justify the value 
of an STC system to all stakeholders and to leverage resources to support the initiative. 

• Horizons LP employs the California Education Research Cooperative (CERC) to 
evaluate its STC system, with the goal of increasing chances of support among 
educators. Their evaluation plan included case studies in contextual and problem-based 
learning in order to demonstrate to educators that applied and contextual/integrated 
learning can increase student achievement and raise test scores. 

• Sierra Regional LP has contracted external evaluators to track program and student 
performance. The evaluation plan was aimed at: identifying focus areas to measure 
program performance, identifying data-collection sources and methodologies, and 
describing a performance model for analyzing process, outcome and continuous 
improvement. The evaluation helps build a solid foundation for sustainability by 
placing performance measures in place and formulating areas of responsibility for 
specific stakeholders. One of the recommendations for sustainability was to improve 
participation by the administrators at the Site Team level (pg. 15). Administrators are 
encouraged to meet and communicate directly with the Site Team on a regular basis. 



Leveraged funding 

Businesses, community-based organizations, post-secondary programs, and other programs 
closely linked to STC may be important sources for alternative funding, both in the near future 
and in the long term. Some LPs have received in-kind and matching contributions as initial 
investments. Others are building on existing programs to capitalize on alternative funding. The 
financial viability of STC will, in large part, depend on LPs’ capacity to gain resource 
commitments from stakeholders to maintain relationships between schools, students, businesses, 
parents, and the community. Sustainability of STC through alternative funding sources will 
necessitate the formation of permanent links between education, businesses, and the community. 

• In Kern County, they have expanded their Texaco Summer Externship program 
primarily with commitments from Texaco. “Texaco hires interns every summer . . . 
This summer STC will pay for three teachers to develop school-based curriculum and 
activities to correspond to work-based learning. Texaco pays $189,750, while STC 
contributes only $14,919.” 

• The Long Beach STC Consortium has created a variety of funding arrangements to 
coordinate funding between STC and partners such as the Youth Job Start Center. The 
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Long Beach School District, for example, will independently fund one of the STC 
coordinator positions during the fourth year of implementation. 

• Charles Schwab & Company, Inc. is a leader among U.S. companies in STC and 
workforce development. Through the company’s foundation, over $500,000 in 
education grants and employee matching funds are donated to local schools 
(kindergarten through post-secondary) and education-related initiatives each year. The 
company has established STC managers, and they set a goal to have over 500 students 
doing meaningful work in the company by the end of 1999. Charles Schwab’s 
commitment focuses on strategic educational LPs, technical assistance, and iimovative 
workforce development practices. 

• The EASTBAY Learns Partnership has been successful in obtaining alternative funding 
from local industries and agencies, as well as building on existing programs for 
financial support. 

• The Contra Costa County Office of Education has committed funds for a full-time STC 
Coordinator well into the 21st Century. Foundation funding was secured to establish 
and coordinate an East Bay Health Careers Learning Collaborative that will supplement 
the four existing Learning Collaboratives coordinated by BaySCAN. All of their work 
directly relates to sustainability of STC in the Bay Area. These funds are available to 
pay students for internships connected to STC. Combined funding from the four Private 
Industry Councils (PICs) exceeds $300,000. Several schools, community colleges, and 
regional occupational programs are using Tech Prep, Perkins and Workability funds to 
further STC. Regional Workforce Preparation & Economic Development Act funds 
continue to support Business-Education liaisons and Pathfinder development in the 
East Bay, as well as inter- segmental Information Technology curriculum development. 
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Summary of Local Partnership Sustainability Efforts 

In quarterly reports to the state, LPs described a variety of strategies for sustaining STC efforts. 
These strategies are listed below. It bears repeating that in many instances, activities that LPs 
identify as “sustainability efforts” are identical to STC implementation activities. 

• Recruiting representatives from key stakeholder groups (including local businesses, 
community-based organizations, and post-secondary institutions) to assist in curriculum 
planning, to organize and support work-based learning, and to provide career guidance. 

• Creating effective systems for communication and collaboration between stakeholders. 

• Securing ongoing resources and long-time commitments from local employers, 
particularly work-based opportunities, as well as finding alternative funding sources. 

• Working with post- secondary institutions to ensure the continuation of students’ 
pathways, to guarantee recognition of students’ acquired skills and certificates, and to 
enrich awareness activities. LPs have strengthened connections with post-secondary 
institutions by creating articulation agreements that allow concurrent high school and 
college credit and by developing sequences of courses that begin at the high school 
level and continue at the post-secondary institution. 

• Some LPs have used data collected through local evaluations to show the community 
the positive impact of STC and justify the value of investing in STC efforts. 

• Providing professional development for stakeholders about the goals and strategies of 
STC, appropriate curriculum and instructional techniques, and administrative aspects of 
STC management such as grant writing. LPs have provided professional development 
opportunities to teachers, administrators, and the business community. 

• Creating databases with directories of employers and available work-based positions, as 
well as interactive links to other career-related sites or other educational institutions. 

• Collaborating with other career-related programs, such as ROP/C, Tech Prep, and 
JTPA. 
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What is Known about Existing Evaluation Data 

We now have large amounts of data about STC in California (see Section III). However, they are 
based on few common measures or reporting formats; are primarily qualitative; and focus on 
processes/plans implemented by LPs rather than student outcomes. Given that LPs are, for the 
most part, building systems and programs, collecting these data is appropriate. However, at 
present, data about STC in California do not provide information that is of highest interest to 
funders and the Legislature (i.e., student performance data). Nor do they allow us to answer most 
key research questions as listed in California’s RPP. The specific findings about existing data on 
California’s STC efforts to date are described below. 

STC’s Diversity of Implementation in California 

As described in Section III, STC in California is notable for the diversity of approaches to 
implementation at the local level. This diversity of implementation influences the nature and 
level of STC impact. Where students participate in regular, intense, multi-year STC activities 
(i.e., in career paths and academies) we might reasonably expect to see meaningful changes in 
test scores, GPAs, and attendance that could result from the STC intervention. However, where 
STC implementation is sporadic and primarily focused on career awareness, as is often the case, 
it is not reasonable to expect meaningful changes in traditional student performance measures 
such as scores from standardized achievement tests. Moreover, STC in California and at the 
national level often does not directly target traditional student performance measures. Instead, 
STC efforts intend to increase students’ capacity to make informed choices about secondary and 
post-secondary education, and employment. Consequently, measuring the full impact of STC (on 
students, teachers, employers, and systems) requires both attitudinaP® and performance measures. 
And, the performance measures targeted for evaluation must be appropriate to STC goals and 
scope of intervention efforts. 



Altitudinal data can be collected through surveys, interviews and focus groups and might include questions related 
to employers', and teachers’ beliefs about the value of STC, student’s beliefs about their future career plans, etc. 
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California ’5 Student Performance Databases 

On a statewide basis, California maintains school-level, not student-level, data about student 
performance. For example, because of privacy requirements, data are available on how well a 
given school performed on a standardized test but not on how individual students in that school 
performed. Even if we can make meaningful distinctions among students in terms of their levels 
of STC participation, it would be difficult to obtain and analyze statewide student performance 
data to correlate with STC exposure. 

State Evaluation Requirements for LPs 

In recognition of the great diversity in implementation approaches, the state developed reporting 
formats that allow LP self-evaluations to focus primarily on completion of implementation plans 
rather than on STC impact. Furthermore, there is a lack of common understanding of STC- 
related concepts and vocabulary for LPs. This is evidenced in the LP plans, in which different 
LPs list similar STC activities under different goals. This lack of common concepts and 
vocabulary exacerbates evaluation difficulties associated with a lack of common measures across 
LPs. 

Available Quantitative Data 

The Progress Measures Survey and LPS provide quantitative data from a substantial number of 
LPs. However, these data focus on STC activities and participation, not student performance. 
From these data we know, in broad terms, what STC opportunities are being offered to whom, 
and who is participating in LP governance activities. As yet, we do not have survey data 
available for some of the newer LPs. For example, there are now 59 LPs in California. In 1997- 
98 there were 28 LPs, 23 of which responded to the LPS. Also, while a small minority of 
independent LP evaluations address student and other impacts quantitatively, they generally did 
not use common data-gathering instruments, and measures, and did not administer the 
instruments in the same way at the same time. 

Finally, California’s response rate to STW-related surveys is notably low. Specifically, 
California’s response rate on the 1998-99 Progress Measures Survey is 53.3%, with only 24 of 
45 LPs responding. Only four other states out of the 45 participating states have lower response 
rates. Moreover, of these four states, one is not structured to report by partnership, two no longer 
receive federal STW funding, and one has returned 100% of data from the state level, but only 
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40% of information at the partnership level. We have been unable to identify any structural, 
fiscal, or other reasons for California’s poor survey response rates. 

Lack of Specificity About Depth and Quality of STC Activities 

Available data on STC implementation in California lack specificity with respect to either quality 
or intensity of STC activities. Some LPs have chosen to focus on intensity — ^to ensure that 
students receive sustained, comprehensive STC exposure. Many others decided to emphasize 
broad coverage. They gave small amounts of funding to a large number of schools to implement 
a variety of one-time and less complex activities. With the exception of Progress Measures and 
LPS data, we do not know, on any consistent basis, how deeply activities were implemented or 
the quality of the activities. Nor are there commonly held definitions of what constitutes the 
depth or quality of activities. Additionally, we do not know about the impact of activities on their 
target audiences. One of our major data sources is the set of quarterly reports submitted to the 
state by LPs. These reports are almost entirely descriptive and lack information on the scope of 
depth of activities. For example, in its quarterly report, one LP stated that it implemented a 
“mini-grant process to provide incentives to teachers to develop innovative projects.” Lacking is 
consistent information on numbers of grants awarded, numbers of projects, and numbers of 
students involved. Another LP cited an activity designed to “increase awareness of and provide 
resources to school sites about work-based activities.” While these LP reports tell us that an 
activity occurred, we cannot make meaningful inferences beyond that occurrence. 

Existing Data and the Research Questions 

The research questions posed in the STC evaluation RFP address sustainability and student 
preparation for post- secondary education and career entry. Regarding sustainability, no standard, 
measurable definition currently exists against which LPs have provided data. Similarly, we do 
not have reliable data about individual students that allow for meaningful statistics or inferences 
about student preparation for post-secondary education and career entry. However, efforts are 
underway to establish a standard definition of sustainability and student data will be collected 
during Phase II of this evaluation. 
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What is Known about STC Implementation 

While existing data have important limitations; particularly with respect to STC impact and 
student outcomes, meaningful information on STC process and implementation can be gleaned 
from them. Here is a summary of the key findings regarding STC process and implementation. 



1 . Diversity and richness of activities are salient features of California ’s STC context. Like the 
rest of the nation, California ’s approaches to STC vary widely. Consistent with the STWOA, 
diversity starts with LP governance and overarching implementation strategies, and includes 
both school-based and work-based activities. For example, at the governance level, 
educational organizations lead almost half of California LPs. Some LPs ensured that 
employers comprised the majority of decision-makers and that employer concerns were the 
first point of reference in determining LP activities. However, 30% of LPs were unwilling to 
designate a single lead organization. This suggests that a wide range of organizations play 
important roles in LP governance. Regarding school-based activities, some LPs and/or 
schools emphasized simple, one-time career awareness events, such as career days or guest 
speakers. Other LPs emphasized intense, often comprehensive school-based experiences, 
such as integrated curriculum. California’s diversity of STC approaches makes sense for this 
large state — STC was designed to be broad and to align with other education reforms (e.g., 
standards-based reform, community service, block scheduling). 

2. Ideally, STC's reach extends from kindergarten to college, across all career-technical and 
academic disciplines, and into the workplace. Such a broad-based reform that targets all K- 
16 students in and out of the workplace must be highly flexible, particularly in its nascent 
stages. California’s approach to STC implementation recognizes and supports local control. 
The state mandated that LPs must achieve certain loosely defined goals but let LPs have wide 
latitude about how, and to what level, those goals would be achieved. Also, as will be 
discussed in depth, the state gave LPs great latitude in how to carry out evaluations. 

3. STC implementation occurs amidst, and often in concert with, other reforms (e.g., WASC 
Focus on Learning, site-based management, higher academic standards. Success for All) that 
may have an impact on student performance outcomes and attitudes. Intensity of STC 
implementation often occurs at levels where it is not possible to meaningfully distinguish its 
impact from that of other reforms or school effects. 

4. Although STC is spread across all levels of K-16 systems and many types of educational 
institutions, data from the LPS indicate that the more intensive aspects of STC are reaching 
relatively few K-16 schools and students. For example, 52.3% of high schools offer some 
t)Tje of career major, a complex STC activity. However, slightly less than 6% of 12“' graders 
had selected a career major by the time they were in their final year of high school. 

5. STC participation is strongest at the high school level. Not surprisingly, involvement with 
LPs is strong at the high school level, falls slightly at the middle school level, and declines 
dramatically at the elementary level. While the average number of high, middle and 
elementary schools participating in LPs is about the same (15.9 high schools, 14.7 middle 
schools, and 13.5 elementary schools), California has approximately six elementary schools 
for every high school. 
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6. LPs are implementing activities that should result in progress toward all California ’s goals 
listed in its Performance Matrix. Analysis of Progress Measures data, LPS data, LP quarterly 
reports, and LP self-assessments provide substantial evidence that LPs are addressing all of 
the goals, albeit with varied activities and levels of success. 

7. School-based activities are more prevalent than work-based activities. Although school- 
based activities outnumber work-based activities, both types of activities are available 
throughout the state. 

8. One-time career development activities appear to be the most prevalent form of STC in terms 
of number of activities and student exposure. Data from the LPS, local LP evaluations, and 
LP quarterly reports all suggest this finding. 

9. Low-intensity school-based activities (one-time events such as career awareness days and 
guest speakers) are more prevalent than high-intensity activities (academies, career paths, 
integrated curriculum, curriculum aligned with an internship). 

\0. Low-intensity work-based activities (job-shadows, worksite visits) are more prevalent than 
high intensity activities (workplace mentorships, unpaid internships). 

1 1 . According to the LPS, schools, employers, and labor organizations are implementing many 
best practices. 

12. STC efforts include different subgroups of students within the population. These include 
minority students, English Language Learners, students with disabilities, students who are 
economically or educationally disadvantaged, pregnant or parenting students, out-of-school 
youth, students pursuing occupations that are nontraditional for their gender, and 
academically gifted and talented students. 

13. California’s efforts in STC tend to mirror those observed in other states. The data from the 
Progress Measures and surveys show that the range of STC activities in California and the 
relative emphases on particular categories of STC activities are similar to those observed at 
the national level. 



Implications of Findings for Evaluating System Building and Sustainability 
Overall, our analyses of implementation, data quality, and data availability indicate that the 
Phase II evaluation must strike a balance between (1) canvassing LP activities across the state to 
achieve a broad understanding of California’s STC progress and impact and (2) delving deeper 
into a more limited number of LPs to better understand systemic change and sustainability of 
STC. Moreover, the complexity of the three identified research questions in the STC evaluation 
RFP (How has STC affected student preparation for post-secondary education and career entry? 
To what degree and in what ways has STC contributed to systemic change? Have STC principles 
penetrated the community deeply enough to be sustainable?), in combination with the paucity of 
comparable, reliable data, demands close scrutiny of both qualitative and quantitative data. For 
example, “systemic change” may manifest itself across LPs in different ways and to different 
degrees. It might take the form of career paths across many high schools, or the 
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institutionalization of employer involvement in curriculum development. Similarly, sustainability 
may involve ensuring that strong business-education partnerships continue after federal funding 
ceases. It may also take the form of ensuring that instructional practices (e.g., delivering 
integrated curricula, teaching to skill standards) have spread across many schools. 



Additional implications include the following: 

1. Diverse STC implementation strategies suggest the need for some flexibility in data 
collection as well as common protocols/measures for Phase II evaluation activities. 

2. Diverse implementation strategies aiming for impact at multiple levels (student, teacher, 
employer, local system, state system) necessitate evaluation at a variety of levels. 

3. Because many STC activities aim to change students’ attitudes as well as skills/knowledge, 
attitudinal measures are appropriate in conjunction with performance and other outcome 
measures. 

4. Seeking correlations between STC activities and changes in student performance or attitudes 
in places where STC activities have been minimal (e.g., one Career Day over the course of a 
year) will yield minimally useful information. Therefore, evaluation activities need to focus 
on LPs where STC activities have been substantial. This suggests that we devote significant 
evaluation resources to data collection at the high school level and to LPs with relatively high 
levels of implementation. 

5. While there has been limited implementation of complex school-based and work-based 
activities, the evaluation must focus on these activities since they have the greatest chance for 
measurable student impact. 

6. Given that existing data offer little to no comparability, standardized, well-defined indicators 
must be developed for Phase II. Similarly, standardized, well-defined data-collection 
instruments and methods are necessary. 

7. Given that many LPs, particularly new ones, are attempting to build systems, rather than 
isolated programs evaluation of system change is critical. 



ERIC 



91 



73 



IV. Implications of Findings for the Phase II Evaluation 



Summary 

The extraordinary diversity of LP implementation of STC, the broad focuses of STC activities 
that often aim for impact on' student attitudes rather on test scores and other traditional measures, 
and the limitations of currently available data do not allow for strong inferences about the status 
of STC in California relative to the research questions of sustainability and student preparation, 
or about how well LPs are meeting their goals. However, while existing data have important 
limitations, particularly with respect to STC impact and student outcomes, meaningful 
information on STC process and implementation can be gleaned from them. Upcoming 
evaluation activities must strike a balance between (1) canvassing LP activities across the state 
with standardized data-collection instruments to achieve a broad understanding of California's 
STC progress and (2) delving deeper into a more limited number of LPs to better understand 
systemic change and sustainability of STC. 
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Building from the observations presented in the previous sections, this section presents the 
statewide evaluation plan for Phase II, the heart of which is an in-depth investigation of the 
sustainability of STC in California. Our attempt in this White Paper to define the baseline status 
of STC activities in California highlights the complexity of the California STC landscape. It also 
highlights the significant challenges inherent to evaluating California’s progress and potential for 
sustaining a statewide STC system. At the same time, the great variety of the STC activities 
across the state described in this White Paper suggests that there is substantial fodder for the 
Phase II evaluation of the impact and sustainability of STC in California. 

This section begins with the guiding principles for Phase II of the statewide STC evaluation. 
Next, the evaluation methodology is described, including the evaluation components, data 
sources, and preliminary analysis strategies. Finally, reporting of evaluation results is discussed. 



Guiding Principles for the Phase II Evaluation 

It is important up front to make explicit the assumptions or guiding principles that underlie an 
evaluation plan, particularly for a study of this magnitude. By so doing, interested parties can 
better understand and provide reactions to the rationale behind the evaluation design and the 
selection of particular methodologies. Also, they would have more information with which to 
interpret the results of the study. Five guiding principles drive the Phase II evaluation plan. 

The Research Questions Dictate the Focus of the Evaluation 

As with any statewide evaluation of a major reform effort, this evaluation study must have a 
clear and agreed-upon focus in order to yield maximally useful results. Four major evaluation 
research questions will serve that purpose, driving the evaluation activities for Phase II. These 
major questions, along with corollary questions for each, are presented in Table 2. 

As shown. Question 1 (What is the status of STC implementation in California?) addresses the 
overall picture of STC in California. Interestingly, this initial question is the only one of the four 
major questions not specified in the RFP. We included this as the first question of the evaluation 
study in the belief that a comprehensive evaluation must begin by laying a descriptive foundation 
of STC in California before targeting more specialized questions. In essence, the White Paper 
represents the initial effort at answering Question 1. During Phase II, we will continue the 
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research and synthesis begun in Phase I to offer a comprehensive picture of the status of STC in 
California. 



Table 2. Research Questions to Guide the Evaluation 

(1) What is the status of STC implementation in California? 

• How many schools, teachers, and students are participating in STC activities? 

• Are there regional differences or other discernible patterns in the type or degree of STC 
participation? 

• How does STC implementation in California compare with that of other select states? 

(2) How has STC affected student preparation for post-secondary education and career entry? 

• What types of students are being impacted by STC for post-secondary and employment 
opportunities? What types of students are not being impacted? 

• What is the evidence that students who participate in STC are better prepared for post- 
secondary education and/or career entry? 

(3) To what degree and in what ways has STC contributed to systemic change? 

• What changes in school programs or infrastructure (e.g., new models of career 
development, new career-focused curricula, new partnerships with employers to 
provide work-based experiences for students) are associated with or can be reasonably 
attributed to STC efforts? 

• How are local and state STC initiatives coordinating with other reform and assessment 
initiatives? 

(4) Have STC principles penetrated the community deeply enough to be sustainable? 

• How will services and positions supported by STC funds be supported by other sources 
after STWOA funding ends? 

• Do stakeholders see value in sustaining a STC system? 

• How will existing STC systems function after the end of state and federal support? 



Our approach to Question 2 {How has STC affected student preparation for post-secondary 
education and career entry?) will be to examine available data to ascertain the impact of STC on 
student preparation. However, it is important to acknowledge that it is likely too early to draw 
definitive conclusions about the influence of STC on student academic performance. Any large- 
scale change to a state’s education system, like STC, caimot substantially affect academic 
performance overnight. It takes time to implement new approaches to teaching, new activities 
outside the classroom, and enhanced efforts to connect classroom instructions to learning in the 
workplace. Moreover, there is the added difficulty of attributing any observed change in student 
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performance to a single systemic iimovation. Specifically, the extensive academic standards and 
assessment reform initiative being implemented across the state confounds any student 
performance data, making it difficult to directly coimect any academic gains to STC. Thus, our 
approach to Question 2 will include looking at various types of data that may indicate potential 
STC impact on student preparation for post-secondary education and career entry, such as 
attendance rates, dropout rates, fulfillment of UC/CSU A-F requirements, rates of transition to 
college, as well as scores on standardized tests (e.g., SAT-9). 

For Question 3 (To what degree and in what ways has STC contributed to systemic change?), we 
plan to examine systemic change in the context of school-based, work-based, and coimecting 
structures and activities. What evidence is there of policy and programmatic changes associated 
with STC? 

Question 4 {Have STC principles penetrated the community deeply enough to be sustainable?) 
overlaps somewhat with the question of systemic change, with both questions focusing on 
meaningful change. However, Question 4 goes well beyond Question 3 to address the central 
issue of sustainability. For this final and most critical research question, we will look at both 
structural and financial aspects of STC sustainability. 



The Essential Elements of STC Must be Reflected in the Evaluation 

Table 3 presents the essential elements of a fully implemented STC system. Developed with 
input from the lAP and consistent with the STWOA and current thinking on STC, this list is 
proposed as an operational definition of STC. For purposes of the evaluation, the essential 
elements will be used as the lens through which to examine and analyze local efforts. However, 
each LP may have a different configuration of the essential elements, based on local conditions, 
priorities, and resources. The wide range of local adaptations of STC also leads to different 
definitions of what constitutes an STC student. Therefore, the evaluation approach will not be to 
“check off’ that each LP demonstrates every element on the list. Instead, the list will be used as a 
reference tool in a more holistic approach to looking at local and statewide STC efforts. 

Triangulation of Multiple Methods and Data Sources Will Help Ensure the Validity of the 
Evaluation Findings 

As previously described, one of the major findings of this Paper is the uneveimess in the 
availability, format, and quality of existing data on STC in California. The proposed evaluation 
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strategy must recognize and adjust for this situation. One way is to purposefully triangulate 
different evaluation methods and data sources, avoiding over-reliance on any one method or data 
source. In this way, shortcomings in the data from any one particular source can be mitigated 
through the collection of similar evidence by multiple methods and from multiple sources. The 
specific methods and data sources to be used are discussed in detail later in the section. 

Stakeholders Need Regular Opportunities and Multiple Avenues to Inform the Evaluation 
Process 

Critical stakeholders need multiple opportunities and avenues (e.g., face-to-face meetings, email, 
and conference calls) to provide feedback so that inclusiveness can be efficiently achieved. Also, 
stakeholders are responsible for providing actual data for the STC evaluation. Thus, it is 
important that project staff continue regular communication with the lAP as well as existing 
statewide STC groups (e.g., STC Advisory Council, System Evaluation and Accountability 
Committee, Employer and Labor Involvement Committee). In addition, the recent formation of 
the two-pronged Practitioner’s Panel (i.e.. Steering Committee plus Technical Review 
Committee) will ensure regular input/feedback by the LPs on statewide evaluation-related 
activities. Specifically, the Practitioner’s Panel will review the evaluation plan and all 
instrumentation to help ensure their appropriateness and feasibility. 

The Evaluation Study Must Successfully Balance the Needs to Minimize Burden on LPs with 
Achieving Comparability in Data and Data Gathering 

A comprehensive evaluation necessarily involves heavy data needs. This can result in a 
significant burden on LPs, a major data source for the evaluation study. Project staff will 
minimize the burden on LPs by making maximal use of existing data and avoiding unnecessary 
duplication of data-collection efforts. Minimizing the burden to LPs also requires building in 
some flexibility in the evaluation design to accommodate LP activities. However, the 
requirement to minimize burden must be weighed against the need for comparability of data 
gathering. As the evaluators, we will develop common data gathering methods and instruments 
with broad-based input and feedback from the Practitioner’s Panel to ensure both the 
comparability of resulting evaluation data and the resource efficiency of the evaluation effort. 

Table 3. Essential Elements of a Fully-Implemented School-to-Career System 

School-Based Learning 

• School-based learning about the world of work based on high academic and technical 
standards 
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• Integrated academic and applied/vocational curriculum and instruction developed through 
business-education cooperation 

• Age-appropriate educational and career guidance and counseling services from 
kindergarten through grade 1 6 

• Opportunities to explore and develop awareness of educational and career options 

• Industry Skills Standards and student certifications that reflect high-level academic and 
technical content and are recognized by employers and educators 

• Curriculum development based on current and projected labor market information 



Work-Based Learning 

• Experiences in the workplace, in school-based enterprises, and through simulations that 
expose students to broad career alternatives and offer opportunities for in-depth 
exploration 

• A continuum of experiences that show students the value of progressively higher levels of 
skill and education (e.g., work site visits, internships, community service, job shadowing, 
mentoring, school-based enterprises, youth and registered apprenticeships, cooperative 
learning, and paid employment) 

• Opportunities provided in a safe and healthy work environment that meets all legal 
requirements 



Linkages 

• Activities that connect school-based and work-based learning and allow students to 
correlate work experiences with a variety of academic topics 

• Kindergarten through post-secondary linkages that keep teachers informed of latest 
developments and allow students to progress seamlessly to higher steps of learning 

• Coordination and integration with existing education, workforce development, welfare 
reform, economic development initiatives, programs, and resources 

• Adult and alternative education integrated into LP 

• Coordination and provision of services through One-Stop Career Centers, allowing 
students and teachers greater access to labor market data and career and training 
opportunities 



Stakeholder Involvement 

• Support and participation of key stakeholders in planning, decision-making, and 
implementation 

• Professional development of all relevant stakeholders to continually spread and increase 
knowledge of STC concepts and best practices 

• STC outreach and communications to improve public awareness and develop more 
participation 
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Scope of Partnership 

• Commitment of required and recommended partners, including both time and resources 

• LP uses demographic data to ensure the partnership represents the community population, 
with decision-making responsibility shared among stakeholders 

• Participation of community-based organizations 

• Continuous improvement of the STC system, based on regular feedback and 
collaboration with relevant stakeholders 



All Students 

• Equal access for all students to all STC components (Sections 102-104 STWOA), with all 
students being aware of and able to obtain locally available STC services 

• Specialized support and resources for all students, as needed, to guarantee equal access 
for special-needs or at-risk students 

• All students are prepared for and have opportunities for high-skill, high-wage careers, 
expanded opportunities, and further learning 



Accountability and Sustainability 

• Oversight and communication within the organizational structure of the LP that facilitate 
participation of stakeholders and reinforce accountability 

• Performance indicators (Section 402 STWOA) are collected and used to improve 
planning and implementation 

• Sustainability and expansion of the STC system by obtaining commitments for resources 
from any appropriate source 

• Capacity building, including effective use of technology to enhance learning and deliver 
related services 



Evaluation Methodology 

As specified in the guiding principles, the evaluation will rely heavily on triangulation of 
multiple, complementary evaluation methods in a comprehensive attempt to answer the research 
questions of interest. Specifically, the evaluation methodology consists of four major categories 
of evaluation methods; a systematic review of data from extant measures, project-developed 
survey research (i.e., employer/labor survey and project-specific survey of LP leadership), a 
comparative analysis of California’s STC effort to those of other select states, and longitudinal 
case studies. Both quantitative and qualitative evaluation research techniques will be used across 
the four methodologies. The first three evaluation methods are aimed primarily at cultivating 
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breadth of information. As such, these methodologies will provide the information necessary to 
“paint a picture” of the full range of STC involvement and impact across the state, covering all 
four research questions. In contrast, the longitudinal case studies will provide the opportunity for 
detailed, in-depth information. As such, this method will be used to better understand complex 
issues, such as the impact of STC involvement on student outcomes, the contribution of STC to 
systemic change, and the sustainability of STC efforts. Each specific evaluation component is 
described below, including its purpose, the approach to data collection, and strategies for data 
processing. 



Review of Data from Extant Measures 
Purpose and description 

Just as extant information was analyzed and synthesized for the White Paper, project staff will 
continue to examine existing sources of data for evidence related to this evaluation’s questions of 
interest. Such external data sources include the Progress Measures Survey. 



Roles and responsibilities for data collection, processing, and reporting 

Building on the research performed for this Paper, project staff will continue to be responsible 
for collecting new and/or revised data from the ongoing data gathering efforts described above. 
We will analyze and synthesize the information from extant sources across years to make 
comparisons and look for growth or other trends of STC activity within and across LPs. 



Project-Developed Survey of LP Leadership 
Purpose and description 

The leadership of all California LPs will be surveyed with a project-developed instrument at key 
points during Phase II. The purpose of this survey is to collect targeted information from all LPs 
about their STC activities that is not available through existing information sources. This survey 
is critical because it is the only project-developed data-collection instrument intended for all LPs 
and will provide a common base of key information about the LPs. It will be administered by 
mail twice during Phase II: Spring 2001 and Spring 2002. 
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The project-developed survey of LP leadership will include items that address key aspects of 
each of the four major research questions. Topics will also include those not adequately 
addressed by existing surveys, such as the state rollout strategy and activities; coordination of 
state and local resources; and the incorporation of state and national skill standards into local 
STC. Even so, the survey will be brief — not more than two two-sided pages in length, so as not 
to overburden the LPs. As appropriate, project staff will employ clustering and factor analytic 
techniques to help reduce and interpret the survey data. 



Roles and responsibilities for data collection, processing, and reporting 

Project staff will be responsible for providing advance notice and orientation to the LPs about the 
purpose, content, and schedule for administration of this survey. We will follow up via telephone 
with those LPs who do not respond within the stated timeframe in an attempt to achieve a 1 00% 
response rate. The results of the first survey will be incorporated into the interim evaluation 
report, while that of the second survey will be included in the final evaluation report. The final 
report will also highlight any progress made between the first and second surveys by comparing 
the results of the two surveys within and across LPs. 

Survey of Employer and Labor Organizations 

Purpose and description 

The survey of employer and labor organizations was first administered in May 2000. This four- 
page survey asks employers and labor organizations about their awareness of STC, the nature of 
their participation in STC activities, their working relationships with the LPs, their, perceptions 
about the benefits and challenges of STC, and their willingness to make long-term commitments 
to STC. It was designed to be simple and brief, consisting predominantly of closed-response 
items. (The results of this initial survey will be summarized and presented to the lAP in July 
2000.) This same instrument will be administered to employers and labor organizations again 
during Phase II (Spring 2001) in order to obtain more comparable, comprehensive information 
about change and growth employer and labor involvement in STC. 



Roles and responsibilities for data collection, processing, and reporting 

In order to understand data-related parameters for the Phase II administration of the employer 
and labor organization survey, it is important to first review the processes for the initial 
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administration. A major purpose of the first survey is to provide information to inform the work 
of STC groups, such as the STC Employer and Labor Involvement Committee. A secondary 
purpose of this survey is to collect baseline information on employer and labor STC involvement 
in order to inform subsequent evaluation activities. In preparation for the survey, LPs were asked 
to provide contact information on the employers and labor organizations that they had invited to 
participate in STC activities. They were also asked to rate the level of participation (from 
minimal to substantial) for each of the employers and labor organizations. Project staff followed 
up with many of the LPs to secure this information. 



More than 12,500 employer names were provided from 32 LPs, whereas 100 labor organizations 
were identified by 25 LPs. Given the large number of employers identified, the decision was 
made to send the survey to a 20% random sample. In contrast, the labor surveys were sent to all 
labor organizations identified by the LPs because of the small overall total. 

One option for the second administration of the employer/labor survey would involve a random 
sample of all employers and labor organizations throughout the state. This approach would yield 
a comprehensive picture about how employers/labor across the state view STC. However, this 
approach is dependent on use of an existing and readily accessible data file for employers and 
labor organizations, respectively. Available project resources will not allow for the creation of 
these databases specifically for this project nor are project resources sufficient to conduct 
substantial reformatting, manipulation, and error-checking/resolution on an existing statewide 
database system. Moreover, surveying a new, random, statewide sample would not yield data 
against which data from the first survey can be meaningfully compared. We would be comparing 
results from two substantially different universes of employees and labor organizations. 

A second option, given the resources of this project, is to cast the second survey administration 
as a follow-up to the first survey administration. Specifically, we are proposing a two-pronged 
survey approach for employers: (1) re-survey of the actual respondents to the first survey and (2) 
survey of another random sample of employers from the existing project database of employers. 
For labor organizations, we propose: (1) a re-survey of respondents to the first survey and (2) a 
survey of additional labor organizations identified by the LPs involved in the case studies. 
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In addition to cost efficiency, there are several other benefits to this proposed approach that make 
it preferable to the first option described above. First, it includes a focus on the subpopulation of 
employers that have some exposure and familiarity with STC. Surveying a sample of all 
employers in California may not yield information as useful as targeting those that have 
familiarity and can provide information about what works and doesn’t work from an informed 
perspective. Second, this approach facilitates the examination of progress over time by 
surveying the same respondents twice, with a reasonable amount of time separating the two 
survey administrations. Finally, it minimizes the burden to LPs by making maximal use of the 
information they have already provided. As described, the only new information request would 
be of case study participants (who are expected to provide in-depth information to evaluators and 
are receiving additional funds to do so) for additional labor contacts. The rationale for this 
request is that, unlike employers, the number of labor organizations identified is very low. 
Hence, we would like to increase the numbers in as efficient a manner as possible. 

As with the first employer/labor survey, project staff will be responsible for administering the 
survey, following up with non-respondents, data processing, and data analysis. 



A Comparative Analysis of California’s STC Effort with Those of Michigan, Oregon, and 
Florida 

In an effort to go beyond California data to understand STC in California, we will examine the 
performance of three exemplar states representing STC environments and strategies particularly 
relevant to California (Michigan, Oregon, and Florida) and will track the progress of STW in 
these three states. While most of the California STC evaluation will examine California relative 
to the goals it has set for itself, these comparisons, using available, comparable data (Progress 
Measures and LPS results), should illuminate how California is performing relative to other 
successful states. We should better understand California’s progress with regard to; 

• provision of STC experiences/activities to students; 

• employer involvement: 

• system growth; 

• sustaining efforts after federal funding; 

• implementing STC systems aligned to standards-based practice; 

• implementing STC in diverse economies, specifically industrialized and non- 
industrialized; 
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• implementing STC in differing levels of urbanicity (urban, rural, and suburban); and 

• implementing STC in multicultural, multilingual regions. 



Michigan, Oregon, and Florida were selected for this comparative analysis because each of these 
states exemplifies successful aspects of STW and is demographically, industrially, and/or 
economically similar to California. Michigan and Oregon represent two states considered 
advanced in implementing STW initiatives in both industrial and non-industrial sectors. Both 
enjoy strong industry and statewide support and have demonstrated success in sustaining 
components of their efforts after startup federal STW funding ceased in 1999. Florida is a state 
demographically similar to California, with success in implementing innovative STW initiatives, 
including successful activities in high-poverty areas. 

The comparisons will center on key indicators from the Progress Measures and LPS that address 
system growth, sustainability, and availability/spread of STC student experiences. Examples of 
these indicators are: amount of LP funding from non-STWOA funds, number of employers 
offering work-based positions, number of students receiving integrated curricula, and number of 
schools offering career development classes. In addition, telephone interviews with key 
administrators in each state will be conducted to obtain information on each state’s STW rollout 
strategy and activities. 

Should California compare relatively well, we have an important indication that California’s 
strategy is working. Should California compare unfavorably, it may be worthwhile to investigate 
why, and to identify what modifications to consider. 

The following discussion provides an overview of each state’s STW initiative and the goals they 
hope to accomplish. 

Michigan 

Michigan is successfully implementing STW activities across the state, including its 
industrialized regions. Michigan’s STW system is coordinated by the Michigan Department of 
Education (MDE) in conjunction with the Michigan Jobs Commission (MJC), a group of state 
policy makers and business representatives. This strategic relationship has allowed for the 
successful implementation of STW activities in various regions throughout the state over the past 
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few years. In 1998, the MJC and MDE began assisting 25 Workforce Development Boards 
(WDBs). Building strong linkages with the employer community is the basis of the WDB 
system. To ensure this, more than half of WDB members are from the private sector. This 
requirement guarantees that the business community’s perspective directs the implementation of 
STW programs. Other states, like California, do not require its LPs to have such a high degree of 
employer involvement. Participation from the Michigan private sector is strong, and stakeholders 
are widely represented in leadership positions. Legislative support for work-based learning is 
such that businesses receive $2,000 in tax credits for providing registered apprenticeship 
opportunities. 

Michigan’s STW model emphasizes the close interaction of two defined parts, LPs and 
programs. Furthermore, it emphasizes career exploration and the transition to work. 

• Partnership: The STW effort must be a collaborative effort between education and 
business, based on the concept of providing all students with career exploration and 
work-related experiences. 

• Programs: STW officials must develop and implement specific programs, services, and 
activities offering students a comprehensive look at available career choices. A 
curriculum must be developed that supports both career exploration and experience and 
includes the following: job shadowing; counseling; career days; tours; work-based 
training; mentoring; and co-op work experience. The inclusion of a staff development 
component is imperative to provide educators the exposure to the needs, skill levels, and 
requirements of the business world. 

State support to WDBs has been substantial. State staff provided technical assistance, field visits, 
teleconferences, seminars, and workshops to the network of LPs to assist LPs in their 
implementation efforts. A career exploration curriculum was developed in 1996 to help teachers 
introduce career awareness into their classes. Furthermore, a curriculum framework incorporating 
career majors and work readiness skills was distributed to all schools in 1997. 

STW implementation in Michigan is steadily progressing. During the last five years of 
implementation, the state has created 45 LPs, including all labor market areas in Michigan. All 
schools are required to include STW content in their school improvement plans. Last year, 
Michigan received additional funds amounting to more than $2.6 million to support its statewide 
STW program, particularly the strategies that will help sustain the program after federal funds are 
exhausted. To facilitate a transition after the federal funding period, the state has committed $23 
million to fund Career Prep activities, a continuation of STW activities. 
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Michigan ’s STW goals 

Michigan established clear, ambitious goals to be reached by the graduates of Class 2000 and 
may be considered a model in its plans for documenting impact and attaining sustainability. In 
comparison to California’s goals, Michigan’s are quite prescriptive and measurable. Key 
Michigan goals are: 

• 1 00% of graduates will have at least one job shadowing experience during grades 8-10. 

• 100% will have an Education for Employment Development Plan. 

• 90% will have an endorsed high school diploma. 

• 50% will have participated in a structured, paid work-based learning experience. 

• 40% will have completed a career major linked to a community college associate degree 
program or registered apprenticeship. 

• 30% will have earned a skill certificate in a career field. 

• 100% of Michigan citizens will understand the goals and elements of the STW Initiative. 

• 50% of Michigan high schools will offer interdisciplinary career majors. 

• 25% of Michigan employers will be actively participating in work-based learning 
options.^’ 



Oregon 

One of the first eight STW implementation states, Oregon is considered one of the national 
leaders in STW implementation. Oregon’s STW initiative began in 1991 with the passage of the 
Oregon Educational Act for the 21®‘ Century and the Oregon Benchmarks. These ambitious 
statutes continue to be sustaining forces behind efforts to develop a standards-based educational 
system that will prepare all Oregonians for productive and successful futures. Together, they call 
for sweeping educational reform, including the implementation of a comprehensive statewide 
student certification system for Oregon’s high school students. 

Similar to California, Oregon created a basic framework for its statewide STW Opportunities 
System rather than establishing a specific program model. The system is based on flexibility and, 
like California’s overall reform efforts, on high standards. Oregon’s STW system development is 
built around eight fundamental elements. These serve as a framework for schools to design and 
implement STW programs while encouraging creative planning and application. The eight 
essential elements are: 



Michigan STW Opportunities- 199 8 Profile. 
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1 . Provide career awareness at the elementary and middle school levels that introduces students 
to the world of work. 

2. Provide career exploration and counseling for students pursuing the Certificate of Initial 
Mastery (CIM) — culminating around grade 1 0 — ^that exposes them to a variety of career 
options through work site experiences. 

3. Provide structured work-based learning opportunities for students pursuing the Certificate of 
Advanced Mastery (CAM), utilizing individual plans that guide and assess students’ 
experience at the workplace. When possible and appropriate, these work-based learning 
experiences will be paid. 

4. Integrate and coordinate academic and occupational instruction as well as school- and work- 
based learning. 

5. Recognize diverse needs of students and provide multiple learning strategies. 

6. Award credentials for both academic and occupational skill mastery that are recognized by 
post-secondary institutions and employers. 

7. Employ a governance infrastructure that represents a broad coalition of employers, teachers, 
students, parents, local policy leaders, and labor. 

8. Conduct continuous evaluation to measure program effectiveness as a basis for modification 
and improvement. 

For most of the state’s regions, initial federal STW funding has ceased, so many LPs have 
already transitioned to almost full local or regional support. However, the ability of local entities 
to provide financial support was cut dramatically by legislation severely limiting the ability of 
localities to collect taxes. Regarding sustainability of classroom activities, Oregon revised the 
implementation timetable for its leading edge Certificate of Initial and Advanced Mastery 
(CIM/CAM). Once the CIM/CAM effort is implemented, schools will offer students occupation- 
based curricula. However, the state delayed CAM implementation until 2004-2005. This reduced 
the pressure to implement STW, especially career majors, and focused the efforts of the schools 
on preparing students for the statewide CIM proficiency testing. 



Oregon ’s STW goals 

Like California, Oregon’s goals are loosely defined and support local control. Oregon’s main 
goal for its STW efforts is to, along with other education and workforce development initiatives 
is to, “prepare its youth for the global competition and changing world of work.” Through STW, 
students will acquire the skills and knowledge essential to compete in the global economy and 
promote lifelong learning. 
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Florida 

Florida has substantial demographic similarities with California — it implements STW in regions 
serving multicultural, multilingual populations. Similar to Oregon, Florida began STW activities 
before it was awarded federal STW funding. Florida’s STW model is designed to address 
specific process, implementation, and evaluation components that ensure a cohesive and high- 
quality systemic effort. It is flexible in application, yet structured to support a consistent 
approach to statewide STW implementation. The model addresses system contents that are 
common to helping both in-school and out-of-school youth. It also connects with the state’s 
school-improvement cooperative education, workplace mentoring, service learning, clinical 
experiences, and school-based enterprises. Finally, the STW model is tied in with the state’s 
school-improvement process, designed to initiate systematic changes that produce innovative, 
relevant instruction in the classroom. 



Florida LPs are required to address the STW components listed below in their plans and are 

asked to describe their strategies for implementing these components within an STW system; 

1. Public awareness involves identifying programs and methods that can promote broad 
political and public support for the regional STW system, as well as provide recruiting 
mechanisms. 

2. Professional development provides capacity-building and developmental training to all 
stakeholders based on existing mechanisms. 

3. Career guidance and counseling ensures that each participating institution implements a 
comprehensive career-guidance and counseling program. Building on the success of Florida's 
legislatively mandated career plan, programs assist each student in selecting an appropriate 
career major. 

4. Support services provide programs for health, human, and remedial-education services to 
ensure the success of all students. 

5. School-based programs build on and expand existing school-based initiatives and programs 
such as Tech Prep, vocational education, and applied academics. 

6. Work-based programs include existing work-based programs and strategies offered under 
Workforce Development, as well as school-based or employer-based models such as 
apprenticeship, youth apprenticeship, pre-apprenticeship, internship, job shadowing, 
cooperative education, workplace mentoring, service learning, clinical experiences, and 
school-based enterprises. 

7. Work site development ensures the identification of work sites and the recruitment of 
employers to provide work-based and/or work site learning experiences for all students. 

8. Placement address the matriculation of graduates into related occupations or continuing- 
education programs. 




107 



89 



V. California STC Evaluation Plan for Phase II 



9. Evaluation and outcomes address the method by which each LP will examine its process and 
assess the product of its activities. Activities implemented by the regional partnership should 
be evaluated by both formative and summative assessments. 

Like California, multiple state bodies (Department of Education, Department of Labor and 
Employment Security, and the Director of the STW evaluation) work together to promote STW. 
Twenty-four Regional Workforce Development Boards (RWDB) have been created using 
existing geographic structures for community colleges (with several areas merged) and for Tech 
Prep consortia. The RWDBs are responsible for reviewing and approving regional LP proposals 
submitted to the state for funding. Twenty-eight regional LPs were funded. 
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Florida ’ s STW goals 

Listed below are Florida’s goals for its STW system. They are similar to Oregon’s and 
California’s in that they are loosely defined and flexible. 

• Higher academic standards: core subjects are emphasized and courses are as demanding 
as they are relevant to students’ lives and the world of work. 

• Reduced dropout rate: showing the relationship between what students are learning in 
school and their career aspirations increases the relevance of learning and reduces the 
likelihood that students will dropout. Students are encouraged — expected — to stay in 
school, meet graduation requirements, and develop into responsible citizens. 

• Improved career opportunities for all: students’ job prospects are enhanced when they 
receive hands-on training along with instruction in employability skills and SCANS 
competencies. 

• More highly skilled workforce: employers gain a more highly skilled workforce when 
schools provide a work-based curriculum and on-site training. 



Roles and responsibilities for data collection, processing, and reporting 

WestEd/MPR will obtain the Progress Measures and LPS data with the permission of Oregon, 
Florida, and Michigan. Project staff will also interview key administrators in each state in order 
to obtain information not available through the national surveys on state roll out strategy and 
activities as well as to clarify and supplement information obtained through the national surveys. 
WestEd/MPR will conduct the comparative analyses, synthesize the results, and present them in 
the two evaluation reports. 

Longitudinal Case Studies 

Purpose and description 

The longitudinal case studies are a key component to the overall evaluation. While the other 
components focus on breadth of information fi'om large and representative samples of 
constituencies, the unique contribution of the case studies is the ability to gamer complex, in- 
depth information. Only through concerted study of selected LPs can the evaluation glean 
detailed information and insight about the inner workings of STC and the nuances of systemic 
change. Most importantly, the case studies will allow us to investigate in depth the ways in 
which STC sustainability is being achieved. 

The strategy is to select 15 to 18 LPs for case studies. As was specified in the Request For 
Proposal for this statewide evaluation study, $2.5 million will be distributed to LPs to conduct 
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the case studies. In August 2000, LPs will be invited to submit a competitive bid for case study 
funding through a new Request for Proposal issued by the lAP. Successful bidders will be 
granted funds to conduct case studies to begin in Winter 2000 and extend through summer 2002 
(anticipated level of funding for each LP ranges from $70,000 to $250,000, depending on size of 
LP and scope of work). 

The availability of such a substantial pool of funds for selected LPs will support intensive data- 
collection efforts at the local level. As will be described later in detail, the LPs themselves will 
have the primary responsibility for data collection and analysis for the case studies, whereas 
WestEd/MPR staff will be responsible for training of LPs, development of instruments, 
monitoring progress, and synthesizing information across the different case studies. Given the 
key role of the case studies to the overall statewide evaluation plan, project staff are working 
closely with the lAP and obtaining input from the Practitioner’s Panel to establish the criteria for 
selecting LPs for the case studies. The final selection criteria will be determined by June 2000. 

Table 4 shows the key sample characteristics and selection criteria that WestEd is proposing for 
the case studies. We believe that a sample of 15 to 18 LPs should be sufficient to address a range 
of STC models of governance and implementation strategies in California. As shown, we are 
proposing that the overall sample be diverse, with representation from different geographic 
regions, types of locales (urban, suburban, rural), sizes of LPs, and populations targeted for 
participation. To achieve this diversity, we will stratify the sample, thereby ensuring 
representation on important variables of interest. 
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Table 4. Key Sample Characteristics and Selection Criteria for the Case Studies Sample 



Key Sample Characteristics 

A. Fifteen to 18 LPs. 

B. Diverse group of LPs. 

• Minimum of 2 cases from each of three types of geographic locales (urban, suburban, 
rural). 

• Minimum of 2 cases from each of three sizes of LPs (number of students served). 

• Diversity in governance models (e.g., decentralized versus centralized, employer- 
driven, school-driven). 

• Diversity in implementation strategies (e.g., mini-grants, professional development, 
work-based learning). 

• Balanced distribution across the state’s geographic regions (north, south, central valley, 
central coastal). 

C. Plurality of high-implementation LPs. 

D. Representation of LPs with medium and low implementation. 

E. Plurality of LPs that are at least two years old. 



Criteria for Selection of Individual LPs 

1. Quality of LP’s previous evaluation efforts, particularly with regard to data similar to that 
which will be gathered for the case studies. LP’s record of completing Progress Measures 
and LPSs, quarterly reports, self-assessments for the lAP, independent evaluations, and 
submission of Employer/Labor Organization data will be considered. 

2. LP commitment to using state-developed instruments and to providing the state with data 
from those instruments. Participating LPs must use state-developed instruments and 
protocols. Comparable data allow for cross-LP and statewide comparisons. 

3. Relevant experience of person(s) who will carry out the case study. LPs selected for 
participation should employ a local evaluator who is experienced in gathering and analyzing 
STC data. LP evaluators should be ready to perform case study duties starting in November, 
2000. 

4. LP capacity to provide student performance data to track a cohort of students and to access 
teachers, administrators, students, parents, employers, labor and organizations. 



In addition, the sample of LPs for the case studies must reflect the wide array of LP 
implementation strategies and provide data necessary to answer our research questions. Given 
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the costs associated with conducting longitudinal case studies, it will be important to pay 
particular attention to high-implementation LPs (i.e., those that are well underway in their 
implementation of STC strategies and activities) in order to thoroughly investigate the conditions 
that lead to success and sustainability. LPs that have not had major implementation of STC 
activities will not be able to give us substantial data about students who participated in STC or 
about sustainability. The return on investment in longitudinal case studies will be insufficient if 
the selected sample expends too many resources focusing on low-implementation LPs. 
Nevertheless, inclusion of some LPs that are struggling with STC implementation is important 
too, allowing in-depth study of the possible barriers to success. Also, preference in consideration 
will be given to LPs that have been in existence for at least two years because older LPs will 
have had the time necessary to achieve system-wide implementation and student participation. 
New LPs, in general, will not have had enough time to reach large numbers of students or to 
move beyond the “start-up” phase. 

As shown in Table 4, other criteria for selection of individual LPs are: the quality of the LP’s 
previous evaluation efforts; their commitment to using project-developed evaluation instruments; 
the relevant experience of the person(s) hired by the LP to conduct the evaluation activities; and 
the LP’s capacity to provide data. These important characteristics will help ensure that the 
selected LPs are successful in providing meaningful data for the statewide evaluation. 

Because of the resources dedicated to the selected LPs to conduct the case studies, the design 
will involve extensive use of both quantitative and qualitative techniques. Planned quantitative 
methods include LP collection and analysis of the following types of quantitative data from 
surveys and existing databases (e.g., CBEDS): numbers and types of schools, teachers, and 
students participating in STC activities; and student enrollment, attendance, and performance 
data. As described previously in this Paper, however, we recognize that there are numerous 
challenges to accessing, collecting, analyzing, and interpreting such data. 

In light of the challenges identified in this Paper related to access, availability, and quality of 
quantitative data related to STC implementation, the case studies will place a strong emphasis on 
qualitative methods. Specifically, selected LPs will use the following qualitative data-collection 
strategies: 
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• Telephone interviews with LP contacts at key institutions. Such interviews will yield 
detailed information about targeted schools, post-secondary institutions, business partners, 
labor partners, etc., and are intended to “pave the way” for on-site visits. 

• Individual and group interviews (including focus groups) with key personnel, 
administrators, business and labor association leaders, teachers, counselors, workplace 
mentors, parents, and students. In some instances, individual interviews will be more 
appropriate to secure in-depth information about LP activities. In other instances, group 
interviews will allow for larger numbers of individuals to express their views in a 
relatively cost-efficient manner. 

Roles and Responsibilities for Data Collection, Processing, and Reporting 
Project staff will be responsible for working closely with the lAP to finalize the criteria for 
selection of LPs, to develop content portions of the RFP (criteria, definitions, scoring rubric, and 
other elements, as necessary), and to review the RFP for LPs interested in participating in the 
case studies. Next, project staff will provide orientation, training, and ongoing 
monitoring/technical assistance to selected LPs on the case study design and data collection. All 
data collection will follow standardized protocols. For example, protocols for site visits will 
provide guidelines, not only on the types of questions to ask of each targeted respondent, but also 
on the number and qualifications of staff that should be assigned to conduct site visits. Such 
protocols will help ensure that consistent, comparable information is collected from all sites and 
will facilitate the organization of data for analysis and reporting. 

Project staff also will be responsible for the development of all common evaluation 
instrumentation (e.g., interview/focus group protocols, classroom observation checklists). Staff 
will then monitor the data-collection efforts of the LPs and serve as a resource throughout the 
duration of the case studies. The WestEd/MPR project team will also have the responsibility for 
summarizing and reporting the aggregate results from the case studies to the lAP. 

The data-related responsibilities of individual LPs that are selected and funded for case studies 
will be significant. They will be responsible for conducting the actual case studies for their 
respective sites, including collecting the data, conducting first-order data analysis (e.g., 
generating descriptive statistics) and reporting results (e.g., data summaries, evaluation reports).^ 
They will also be responsible for timely submission of data to WestEd/MPR staff. In particular, 
LPs will be responsible for meeting two important project deadlines: providing data and analysis 
in time for WestEd/MPR staff to compile, analyze, and then incorporate them into the interim 
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evaluation report and the final evaluation report. As such, LPs will need to collect and submit 
qualitative case study data to WestEd by April 2001 in order to inform the interim evaluation 
report. Given the difficulty in collecting student outcome data, such data will be due in January 
2002, in time to inform the final evaluation report. Below is a list of specific data-related tasks 
that LPs could be asked to perform for the case studies: 

1 . Hire an external evaluator to conduct the tasks described below. 

2. Use data-gathering instruments and protocols supplied by WestEd/MPR, during time periods 
specified by WestEd/MPR. 

3. Collect data from samples drawn from the groups listed below. Staff and evaluators from 
each participating LP will be responsible for obtaining access to schools and/or school 
districts to administer surveys of students and to interview school staff. Substantial follow- 
up will be necessary to achieve desired high response rates. 

• Employers of students who participated in high-intensity STC activities (e.g., career 
majors, academies); 

• Two cohorts of high school seniors (’01, ’02); 

• Follow-up data from the ’01 cohort; 

• Administrators at all high schools in the LP; 

• Administrators at all community colleges in the LP; and 

• Administrators at all middle and junior high schools in the LP. 

4. Complete Progress Measures and LPSs, including student participation rates. 

5. Gather select CBEDS data (e.g., A-F completion rates, advanced math completion rates) for 

the high schools in the LP. CBEDS data will be gathered twice (for the 99-00 year and the 

00-01 year). 

6. Gather select student performance data (SAT-9 scores, attendance, discipline, dropout) at the 
school (not student) level. 

7. Identify and interview employers of high school students. 

8. Identify and interview school site staff. 

9. Gather documents relevant to the impact of STC on student performance and structural 
change. 

10. Analyze and report on all data listed above, using prescribed formats provided by 
WestEd/MPR. 

Clearly, the above list is very comprehensive. It will be the responsibility of individual LPs 
during the competitive bid process to describe which of the above they can and carmot provide 
and why as well as to propose other evaluation tasks that may be of particular interest to them. 




114 



96 



V. California STC Evaluation Plan for Phase II 



Reporting of Evaluation Results 

The results of the evaluation study will be presented in two separate reports: an interim report 
(due June 2001) and a final report (due June 2002). The interim report will summarize the 
evaluation research results through Spring 2001, serving to foreshadow the summative findings 
of the comprehensive final report. 
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Summary 1: The National Evaluation of STW Implementation 



Hershey, A., Silverberg, M., Haimson, J., Hudis P., & Jackson, R. Expanding Options 
for Students: Report to Congress on the National Evaluation of STW 
Implementation, Mathematica Policy Research: Princeton, NJ, February 1999. 
[Hereafter referred to as the National Evaluation.] 

The purpose of this study is to assess the implementation of STW across the United 
States. The major evaluation questions are: 

• Have states and LPs created coherent STW systems of connected, sustainable 
practices and programs? 

• How do STW systems change what students do at the elementary and secondary 
education levels? 

• How do post-secondary paths change as STW systems are developed? 

• Are the activities and practices the STWOA promotes adopted on a wide scale? 



The 1999 report draws on the following sources: 

• 1996 and 1997 LPSs. 

• In-depth case studies of eight states22 with two rounds of visits to 39 LPs, in 
spring 1996 and in spring and fall 1997. 

• Surveys of random samples of 12th graders in eight in-depth study states in 
spring 1996 and 1998, and a fall 1997 follow-up survey of the class of 1996 
sample. 

• Analysis of high school transcripts for the 1996 student sample. 



The in-depth study states are Florida, Kentucky, Maryland, Massachusetts, Michigan, Ohio, Oregon, and 
Wisconsin. 
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Summaries of the National STW Studies 

Summary 2: The STW Progress Measures 

Medrich, E., Merola, L., & Ramer, C. STW Progress Measures: A Report to the 
National STW Office for the Period July 1, 1997-June 30, 1998, MPR 
Associates: Berkeley, California, February 2000. [Hereafter referred to as 
Progress Measures.] 

This annual report, based on a yearly survey sent to all funded STW LPs, provides information 
about the growth of the STW enterprise. It has four objectives: 



• To establish a system of performance measures for assessing progress in meeting 
the objectives of the STWOA. 

• To develop a common language around STW, so that data will be comparable and 
of high quality across LPs and across states. 

• To provide a framework within which states can design their own STW data 
systems for program improvement purposes that range beyond the reporting 
function at the national level. 

• To help state and local STW practitioners develop the skills necessary to measure 
the success of their efforts. 



Results from the 2000 report are based on data collected via a mail-in survey that covered the 
period July 1, 1997-June 30, 1998 and are based on responses received from 770 of the 985 LPs 
(78%) in 34 of 38 funded states. The survey collected information on indicators of STW systems 
development; school and student participation; employer participation; student outcomes; and 
state and local resources beyond federal STW funds. 
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Summaries of the National STW Studies 



Summary 3: STW in the 1990s: A Look at Programs and Practices in American 
High Schools 

Visher, M., Lauen, D., Merola, L. & Medrich, E. STW in the 1990s: A Look at 
Programs and Practices in American High Schools, MPR Associates: 
Berkeley, California, August 1998; [Hereafter referred to as NLSY.] 

This study explores the question of whether high schools are changing in directions envisioned 
by the Act, regardless of whether they were funded by the STWOA. In the 1996-1997 school 
year, a large sample of high schools (approximately 25% of all U.S. high schools) were surveyed 
for the National Longitudinal Survey of Youth (NLSY97), a study designed to document the 
transition from school to work and into adulthood. The school survey, completed by high school 
administrative staff, describes the characteristics of STW programs and practices such as career 
majors, job-shadowing, and internships, in addition to other important school programs, 
characteristics, and emphases. Seventy-two percent of the eligible schools returned completed 
questionnaires. 




121 



appendix C 



Performance Matrix and Out-Year Performance Matrix 



Available Upon Request 




122 



Analysis of Performance Matrix 

^HENSIVE LOCAL SCHOOL TO-CAREER SYS! 






-i 

< 

O 



(/i 

a 



U 

o 



u 

o 

.D 

-d 

o 

{/i 

O 

►PN 

DD 

o 

A 

o 






o 

a 

g 

A 

IK 



3 

c/T 

cd 

x: 

4-* 

cd 

Dh 

L-i 

<u 

<u 

o 

d 

0 

4— • 

cd 

feb 

<u 

4-^ 

d 

1 

1 



■o 

(U 

4-^ 

s 

*5 

d 

Cd 



<u 

6 

Dh 

o 

"S 

> 

<u 



"O cd 



cd 

1 2 P 



o 



0 > d 
^ <D 
cd 3 

H ^ 

^ c/^ 



■S) 



W) 

d 

1^ 

cd 

<u 



a 

CJ) 

o 

o 

<u 



<u 
W) 
<u 
o ^ 



c/^ 

H <u 
ti -c: 
^ >o 
cd 

o ^ 

cd 

Td 
d 
cd 



cd 

O 

W) 

P 

<U 

4-^ 

Cd 

feb 

<u 



O) 

x: 

'4-J 

Cd 

Oh 

<u 

o 

<u 

4-^ 

cd 

(U 

D 



& 

o 

p 

y) 

<u 

CO 

cd 

<u 

u-i 

o 

d 



CO 

<D 

O 

§ 

5 

Dh 

X 

<u 

p 

Xi 

d 

(U 

s 

6 

0 

1 

Dh 



<u 

;-i 

3 

a" 

<u 



CO 

O 

Dh 

<u 

d 

(U 

4-^ 

cd 

(U 

U 



P 

y^ 

o 

4-^ 

ro 

<u 

4-* 

s 

"S 

;-i 

CO 

(U 

CO 

;-i 

d 

o 

o 

<u 

Dh' 

o 

"S 

> 

<u 

p 



CJ) 

■o 

cd 

X 3 

yi 

O 



S 

M 

■o 

cd 

o 

cd 

CO 

cd 



M 

■o 



<u 

x: 

cd 

d> 



cd 

O 

W) 

p 

y^ 

o 

4-^ 

(U 

4-^ 

2 

- § 
lS- 

CO .y 

[| E 
ro 3 
3.S 

4-^ 

CO ^ 

<U ^ 
O 

S <1^ 

cd 

P ^ 
C ^ 
O ^ 

S 

^ P 

Dh ^ 

o ^ 

^ S 

"O -o 

O S 

4-^ ^ 



(U 

x: 

cd 



Dh 

O 

"S 

> 

■o 



CO 

(U 

> 

4-^ 

<u 

o 

C 

(U 

■o 

> 

o 

Oh 

O 

4-^ 

CO 

CO 

<D 

O 

O 



CO 

2 

O 

C 

(U 

4-J 

<u 

Dh 

s 

o 

o 

y) 

Z 

< 

u 

y) 

o 

4-^ 

<u 

4 -* 

2 

"S 



& 



CO 



"O X 3 



P P 



Oh 

P 



^ y) 



x: 
8 
a 

<u 

4-^ 

Cd 
(U 



CO 

<U 

4 — • 

lx: 

o 

o 

4 — ► 

c 

o 

CO 

(D 

D4 



O 

o 

Oh 

<D 

<U 

4-^ 

Cd 

<D 



<o 



<u 



■S) 



cd 






<u p 



p 



cd 



cd 



a 



fl 

o 

a 

o 

s 

a 

>% 

y) 



DX) 
fl 

• pN 

c 

■S -a 

■Q ^ P 
^ 

^ O 
CO w ,55 

« j 3 fi 

. l^-S - 

o pfl « 2 

3 L ’Q 






15 -a 

>c 



- -o 

^ a 



>G ^ Z 

« 2 
S A e#) 



DX) 

fl 

'PN 

fl 

U 

Cd g 

fli tap 

■25 S 

s 

o fl 

S 2 

fl 

o 

.® a 



£ 




123 



Analysis of Performance Matrix 



O 

-M 

t: 

o 

(U 

(U 

c 3 

cd 



W) 

c 

I 

(U 

-1 

(U 

■o 



D. 

O 

13 

> 

(U 

Q 



o 

■4— * 

CO 



(U 

o 



C /5 

(U 

o 

c 

(U 

S 

g- 

(U 

(D 

CO 

D. 



(U 



(U 

C /5 

CO 

'S 

T 3 

(U 

'S 

(U 

l-H 



c /5 

& 

e 

(U 

s 

W) 

■o 

CO 

x: 

o 



o 

C/I 

t 

<L> 

c/5 



(U 

o 

5 

B. 

}-H 

o 



C/I 

(U 

c/i 

C/I 

(U 

g 

c/I 

O 

O 

-C 

CO 

C 

(U 

(U 

t 3 

-D 



CO 



(U 

3 

cr 

(U 



CiX) 

G 



G 

<U 

I 

§) 

G 

G 

5 

l-i 

B 

o 

o 

G. 

O 

"S 

> 

d> 



d> 

CO 

G 



d> 

x: 

d> 

■4— » 

o 

e 

2 

Cu 

■o 



o 

G 

bO 

G 

t3 

(U 



CO 

t/I 

d> 

O 

O 

Uh 

Oh 

d> 

> 



CO 

O 



t: 

o 

G 

o 

G 

G 



G 

■o 



O 

o 



(U 

■4— • 
CO 

CO 



(U 

CX 

e 

o 

o 

■S) 

G 

p 



‘ -o 

G 



B 

G 

O 

o 

o 

G 

(U 

B 

2 

D 



& 

o 



■o 

d> 

B 

13 

l-H 

bB 

G 



o 

o 

x: 

13 

CO 



o 

04 

CO 



O 

O 

x: 

13 

CO 

< 



CO 

G 

O 

■G 

G 

■4— » 

G 

(U 

CO 

(U 

;-H 

O 4 

(U 

e 

CO 

B 

(U 

■o 



G 

(U 

O 



G 

(U 

e 

CO 

CO 

O 

G 

G 

1 ^ 

G 



O 

CO 

"O 

G 

G 

CO 

CO 

(U 

G 

(U 

G 

G 



G 

G 

G 



G 

G 

e 

e 

o 

o 

G 

> 

O 



bO 

.s 

G 

d> 



CO 

'G 

d> 

d> 

G 

d> 

(D 

■4-> 

& 

(U 

G 



7 D D 
o 

§ 

S 

O 4 
X 
(U 



O T 3 
^ G 

B^ 
B o 

S ^ 

G .G 

bD^ 
G G 
G.& 

3 .2 

Im g 



G 



(U (U 
G G 
13 o 

G G 
,(U (U 

H H 



B) 



bX) 

G 

O 

G 

(U 

}-H 

(U 

♦-4 

G 

O 

o 

o 

(U 

G 



G 

O 4 

O 

B 

> 

G 



G 

(U 

e 

04 

o 

13 

> 

G 

(U 

bO 

G 

3 

O 

o 

G 

<U 



B) 



(U 

CO 

P 



G 

G 

O 

G 



CO 

CO 

(U 

G 

CO 

G 

OQ 



2 

O 4 

o 

(U 

G 

O 

Oh 



(U 

CO 

G 

G 

O 

o 

p 

yD 



12 [G 

o 

(U 
04 

CO 



G (G 
(D 



CO 

I— ^ 



(U 
G 
(U o 

CO G 

P 



G 

<U 

e 

04 

o 

13 

> 

G 

(U 

G 

bO 

G 

B 



CO 

G 

X 3 

G 

(U 

G 

U 

G 



O 4 

O 

B 

> 

(U 

Q 



o 

0 4 

e 

(U 

G 

O 

G 

B 

G 

bO 

G 

G 

> 

2 

^co 

0 ) 0 ) 

to 'o^ 

CO O 4 

5 G 

2 
G *G 
o G 

w O 

> > 

G G 

2 o 

- S 

O 4 <u 
O G 
T! G 

S? o 

> G 

Q O 



G 

(U 

e 

Oh 

o 
B 
> 
(U 
, G 

e 

G 

, B 

o 



CO 

i 

u 



e 

1 

I 

o 

B 

o 



(U 

G 

G 

(U 

- 4 -J 

" (U 

'B 



(U 

CO 

G 

2 

B 

Wh 

B 

G 

O 

G 

(U 

■ 4 -J 

(U 

G 

> 

P 



O 

(U 

■4-J 

G 

G 

G 

O 



o 

o 

o 

rN 



I 

G 

O 



G 

O 

(U 

- 4 -J 

G 

feb 

(U 

- 4 -J 

G 



G 

<U 

fc 

B 

- 4 -J 

G 

G 

- 4 -J 

CO 

d> 



G 

<U 

(U 

B 

G 



CO 

z; 

< 

u 

(U 

& 

(U 

- 4 -^ 

G 



X 

tG 

B 

u 



x 3 

CO 

G 

O 

B 

2 

bO 

G 

’C 

o 

- 4 -^ 

G 

d> 

B 

04 

o 

B 

> 

(U 

P 



DX) 

fl 



O 
O 

G 

S.2 

o -o .tS 

^ S > 

^ *G 
G B 



G 



O 

G 



G G 



a -2 

s « 

G G 
o o 
G 

-rt o 
S *pG 

2 ^ 

Sd « 

""2 
G A 
-H G 




124 



Analysis of performance Matrix 



Z) 

X 



< 

J 



o 

o 

X 

J 

Z5 

< 

o 

-i 



Z5 

X 



0 

u 

iij 

z; 

o 

c 

o 



Th 

O 

4— » 

■o 



(U 

> 

52 

52 

& 
4— • 

3 

e 

c3 

CO 

}~t 

(U 

(U 

o 

;j 

i/D 

T3 

C 

cd 



d> 

Dh 

e 

o 

o 

O 



(U 

4— » 

C 

(U 

o 

(D 

o 

C 

cd 

■o 

J) 

'SD 



<D 

CO 

§ 

O 

O 

C 

(U 

(U 

q 3 

r) 

c 

o 

4— > 

cd 



o 

o 



cd 



: 

; u 2 









cd cd 



(U 



■o 

cd 

o 

cd 

o 

lc 

o 

0 ) 

C14 

CO 

■S) 

ii 

e 

5 

3 

o 

E 

^3 

O 

o 

CO 

O 

S 4 

o 

(D 

4-> 

2 

3 



o 

52 . 

o' 

;-H 

0 4 

I.H 

o 

5 

o 

CO 

■o 

O 

2 

o 



o 

00 

c 

cd 

s: 

Ij 

X 

o 

00 

c 

% 

o 

3 

o 

CO 

cd 

cd 

TO 

o 

> 



D 4 

O 

o 

> 

o 



o 

C14 

e 

o 

o 

x: 

00 

ll 

4— • 

CO 

■0 



<0 



CO 

0 

O 

C 

O 

■M 

o 

o 

p 

i-t 

o 

CO 

C 

o 

C14 

y) 



^3 

O 

a 

o 

H 

TO 

TO 



00 

c 

O 

CO 

C 

?3 

O 

o 

TO 

c: 

cd 

o 

o 

J 

■o 

§) 

p 

4—* 

C 

o 

e 

u 

H. 

6 



CO 

CO 

<U 

C 

O 

cd 

K 

o 

(D 

cd 

P 

D 4 

o 

o 

> 

o 



E/) 

4.-I 

^3 

O 



00 



§) 

o 

I-H 

c 

o 

•O 

cd 

o 

TO 

X) 

I-H 

o 

x: 

o 

cd 

x: 



2 

4—* 

3 

o 

3 

o 

?3 

TO 

O 

4—* 

;3 

O 

JD 



O 

O4 

CO 



CO 

(U 

O 

C 

(U 

I-H 

.(U 

-I— ( 

c 

o 

o 

I-H 

(U 

X2 

4— • 

o 

TO 



C14 

(U 



TO 



(D 

2 

o 

(D 

> 

cd 



(U 



TO 

d> 

4-J 

3 



cd 

4—* 

y) 



o 

o 

A 

® § 
§ s 
lg 

00 



o 53 _ 

u 'O 

> 2 

isv ^ 

35 o> 'O ^ O 
G> 'O S 
^ C M ^ 



— 5 CO 



0 s a ^ jp « s 
2 5 2 




^ s 



00 

D 

0 

1 ^ 

g « 

A 

^ o 

^ M 00 

fl ^ 

2 0 fl 
a*pO 



er|c 



125 



ANALYSIS OF PERFORMANCE MATRIX 





126 



Analysis of performance Matrix 




o 

ERIC 



127 



Analysis of performance Matrix 



§ 

si 

'r> 

X 

si 



< 

J 

I 

P 

o 

p 

X 

J 

-] 

< 

J 

p 

'si 

> 

z; 

'si 

X 



P 

u 

z; 

P 

< 

P 

P 



(/} 

C 

cd 

Oh 

C /1 

O 

o 

-C 

(/} 

(U 

cd 

(/} 



(U 

(/} 

C 

o 

Oh 

(/} 

(U 

>% 

o 

g 

W) 

uh 

(U 



(U 

<4— » 

o 

c 

< 4 — » 

CO 

■3 

■o 

cd 

T 3 

f 

O 

o 

(U 



'‘i CO hrt 

■o ^ > 

Cli w (U 

D .2 Q 



CO 

>% 

cd 

■ 4 —* 

Cd 

Oh 

uh 

0) 

(U 

s 

o 



Oh 

O 



0 

04 

1 

Oh 

o 

3 

> 

(U 

3 



cd 

o 

w 

3 

p 

Oh 

o 

3 

> 

(U 

3 



CO 

5 

cd 

Oh 

;-i 

(U 

(U 

;-i 

cd 

o 



■o 

o 



(U 

CO 

7^ 

(U 

o 

(U 

CO 

P 

po 

CO 

O 

o 

C 

cd 

> 

W) 

o 



(U 

CO 

O 

o 

jC 

o 



O 

(U 

cd 

d> 



(U 

O 

(U 

p 

Oh 

o 

0> 

O 

IP 

u 

CO 

O 



CO 

O 

(U 

P 

Oh 

O 

<L> 

O 

O 

o 

o 



§) 



(D 

O 

(D 

O 

Oh 

0 

1 

(D 

O 

P 

;-i 

O 

O 



CO 

;-i 

(U 

o 

<D 

P 

<U 

<u 

p 

Oh 

0 

<4—* 

Xi 

1 

(U 

O 

O 

■o 

(U 

CO 

cd 

n 

Oh 

O 

o 

5 

Oh 

O 

3 



y 

O 

CO 

O 

<u 

p 

Oh 

0 

4 — » 

yD 

1 

<L> 

O 

P 

O 

o 

o 

o 

-4-t 

cd 



"a 

2 

Oh 

W) 

o 

aJ 

cd 

E 

u 

cd 

d> 

i_^ 

p 



o 

*-» 

d> 

o' 

o 

■o 



o 

W) 

cd 

Oh 

O 



O 

o' 

O 



■o 

cd 



O 

& 

Oh 

CO 

O 

P 

PO 

o 

o 

X) 

W) 

o 

■ 4 ~> 

cd 

o 

2 

o 

2 

2 

o 

o 



cd 

C 

o 

4 — » 

Cd 

2 ^ 

CO O 

o-S 

2 E 

a, M 
o .CO 

p g 



cd 

w 

J? 

"O 

4-^ 

o 



o 



CO 

O 

2 

CJ 

o 



o 

o 

X) 

c 

o 

4~> 

cd 



o 

■M 

o 

3 CO 
^ O 

^ 2 
-o 3 
2 ^ 

SP’S 

O CO 
O 

fE g 

> 'O 

a B 

^ CO 

2 § 

O 

y CO 



CO 

x; 

cd 

Oh 

o 

2 

o 

o 

■o 

O 

2 

o 

i-H 

2 

o 

o 

O 



O 

O 



CO 

■o 

O 

O 



[) CO 

" O 
J 

o 
i x: 

> -4-^ 

' ^ 
o 

6b 



Cd 

C 

CO 

O 

-o 

o 

W) 

o 

5b 

2 

o 

o 

'H— • 

■o 

o 

2 

o 



,o 

■M 

o 

o 

2 

6b 

o 

-4—* 

o 



o 

o 

4-4 

cd 



■o 

> 

o 

Dh 



CO 

£! 

2 

Oh 

6 

d> 

6 

o 

o 
+- • 
cd 
o 

y 



<D 

2 

O' 

\b 

■4-^ 

o 

Oh 

O 

o 

3 

■4-^ 

X) 



o 



o 

lx 

2 

c 

o 

-a 

o 

CO 

cd 

X 

tS 

■a 

o 

o 

o 

CO 

cd 

E 

3 

3 

o 



g 

■3 

o 



to g 

.(fi O 

■gc 

«’■£ 
te P 
c u 

« -M 

- s 

5 

^ 2 
y) 2 















a 

0 

1 2 
o « 

° u 
W) S 5 
a 



y a 



> 2 ^ 
2 ^ a 
cx£ y 



■o 
o 

A 

Ms 

S &£ 

a 



o 

o 

H 3 

O 

a 

JD 



S 

s g "=2 

■I ® 2 2 

s Q- fl 5 

s 

M - 

o 



’£■3 c 



>r ^ 



ERIC 



128 



ANALYSIS OF PERFORMANCE MATRIX 





129 



Analysis of Performance Matrix 



Z) 

in 

=) 

P 

■fl 

P 

P 

< 

P 

o 

z; 

O 






± 



p 

p 

a 

p 

< 

o 



3 

<u 

x: 

W) 

c 

P-* 

cd 

feb 

(U 



y) 

z; 

< 

U 

y) 

c 

o 

W) 

c 

2 

S 



(U 

cd 

(U 

p^ 

■o 

> 

p 



c 

o 

"5 

0 

■o 

<u 

W) 

p 

s 

1 

c 

o 

<u 

W) 

cd 

l-H 

p 

8 

§ 



2 

cd 

feb 

o 

l-H 

a. 

C 50 

P 

% 

T 3 

P 

-P 

O 

■o 

p 

a 

a. 

X 

0> 

o 5 

p^ 

p 

<u 

1_^ 

D 



a 



d 

o 

p^ 

W) 

p 

p 

2 

p^ 

<u 

o 

§ 

CO 

P 

<u 

T 3 

> 

p 



2 

p 

3 

2 

p 

o 

p 

o 

1^ 

Th 

o 

0 

p^ 

CO 

1 

(D 

p^ 

& 

P 



<U 



o 

d 

P 

y: 

p 

p 

o 

'5 

& 

0 

1 

Ph 

p^ 

p' 

■p 



p 



p 

o 

CO 

3 

S' 

0) 

<u 

P-* 

CO 

(D 

> 

P 

O 

p^ 

& 

IP 

CO 

I-H 

<U 

P 



a. 

tjo 

p 

o 

Oh 

O 

"S 

> 

<U 



p 

<D 

p-> 

P 

O 

CO 



O. 

P 

cr 

<u 

W) 

p 

O S 
P <u 

(U b 
}-H P 

<1^ p- 

c s 

o S 

O CO 

<D ^ 
P »p 
P 

> P 



I 

feb 

S 

Oh 

W) 

p 

C 

o 

p-> 

p 

<u 



p 

S 

d 

p-» 

CO 



<u 

p^ 

p^ 

<u 

o 

o 

1^ 

d> 

o 

Vh 

P 

o 

CO 

<U 

l-H 

P 

yi 

p 

Oh 

O 

> 

<u 



p 

CD 

P 

5 

po 

p 

o 

P 

p 

o 

p 

p 

W 

p 

o 

CD 

Oh 

y) 



p 

3 

<D 

Oh 

CO 



<D 

CO 

O 

P 



P 

P 

P 



P 

O 

p-» 

P 

o 

p 

p 

<D 

P 

3 

<D 

Oh 

CO 



Oh 

Oh 

IP 

CO 

1-H 

CD 

P 



t: 

o 

Oh 

Oh 

P 

CO 

W) 

C 

P 

P 

o 

p 

c/f 

CD 

O 

§ 

CO 

W) 

P 

P 

o 

o 



p ■ 
CD 

2 

CD 

O 

P 

Hh^ 

IXJ CD 
^ CD 
^ P 



JO 

P 

Oh 



P 

CD 

Oh 

O 

P 

|u 

1 -H 

CD 

CD 

P 



0 

1 

CO 



P 

CD 

P 



CD 

P 

< 

o 

-f 



OD 

P 

€ 

Oh 

CO 



OD 

P 

P 

P 

3 

p 



p 

o 

P 

feb 

D 

H-> 

P 



P 

CD 

P 



P 

CD 

> 

? 

P 



P 

"3 

Oh 

X 

CD 

P 

CD 

CD 

O 

CD 

P 

O 

P 



P 

CD 

P 



O 

O 

P 

0 

CO 

D 

> 

'S 

1 



p 

CD 

P 

"3 

l-H 

CO 

1 

feb 

o 

Wh 

Oh 

P 

o 

o 

P 

p 

CD 

3 

2 
Oh 

-M 

o 

p 

o 



p 

CD 

P 

”3 

I-H 

CO 

CD 

5b 

D 

2 



p 

CD 

3 

P 

P 

> 

CD 

P 

P 

P 

CD 

tP 

P 

P 

CD 

O 

CO 

CD 

O 

2 

8 

CD 



CO 

CD 



CD 

P 



Oh 

X 

CD 



P 

' I-H 



Oh 

O 

3 

> 

P 

■M 

■M 

P 

H— • 

y) 



p 

o 

C 

S) 

D 

3 

O 

3 

l-H 

P 

CD 



P 

CD 

O 

P 

P 

> 

hP 

P 

£ 

o 

2 

CD 

2 

P 

P 

CO 

CD 



O 

0 

CO 

1 

p 

I 

p 



p 

CO 

p 

O 

c 

8) 

D 

2 

o 



D 

3 

H— • 

y) 



o 

o 

p 

o 

y^ 

CD 

P 



P 

’3 

CD 

Oh 

CO 

CD 

y) 



fl 

^ ts 

CO fl 

« g 

2 

M o 

CO u 

O 

P 

s 



p 

rv 

a 

M fli 

O 

1“ 

o 

■a ^ 

^ I 

p ^ 




130 



ANALYSIS OF PERFORMANCE MATRIX 





131 



Analysis of Performance Matrix 






"O 

I 

W) 

g 

4— > 

(U 

d> 

6 

(U 

■a 

I 

(U 

4— » 

cd 



CO 

O 

CO 

CO 

(D 

CO 

W) 

g 

g 

is 

-o 

(U 

4— » 
4— » 

CO 

O 



0 
C 

4— > 

CO 

■3 

Uh 

5 

JS 

Ht— * 

c 

1 

i/) 

W) 

g 

(U 

I 

Hh 

6 



CO 

o 

o 

Dh 

o 

> 

(U 



CO 

g 

5b 

(U 

Uh 

4-> 

CO 

O 

4— > 

o 

0> 

o 

o 

cO 

4-» 

,c0 

■o 

4— > 

o 

o 

3 

CO 

c/T 

Uh 

O 

4-J 

CO 

o 

■3 

g 

(U 

4— • 

CO 

(U 

L.*-l 



<J 

5 50 

A U 

§ ® 

s 

*m «pn 

^ 'O 
O S 
Oh a 



<u 

CO 

W) 

g 

3 

CO 

O 

o 

4-1 

o 

(U 

> 

■o 

<u 

x: 

4— • 

■o 

CO 



CO 

d) 

g 

■g 

4~> 

C 

d) 

6 

Dh 

o 

3 

> 

d) 

P 

o3 

d) 

3 

o 

3 

tz; 

d) 

4— » 

CO 

feb 

d> 

4— • 



t: 

o 

Dh 

4— • 

■o 



Dh 

o 

3 

> 

d) 

P 



W) 

I 

"O 

CO 

CO 

Uh 

d) 

x: 

cO 

d) 

.4—* 

d) 

6 

Dh 

o 

3 

> 

T3 



0> 

4— • 

<u 

o 

Uh 

d) 

d) 

0 

1 

4-4 

CO 

XI 



U 
^ H 
fi3 cZi 

c ^ 



o 

2 fl 

« ,2 

s§| 

2 &• w 



CO 

O 

Dh 

to 

CO 



o 

feb 

<D 

4>4 

a 

O 

CO 

d) 



CO 

CO 

p 

rH 

W 

00 

c 



a 

d) 

6 

04 

o 

3 

> 

TO 



X 

d) 

O 

4—* 

g 

4-H 

+-4 

cO 

3 

P 

y) 

d) 

4-4 

CO 

feb 

d> 

4— • 

a 



Oh 

O 

3 

> 

d) 

3 



g 

S 

4—* 

o 



d) 

CO 

CO 

4— • 

,cO 

■o 

d) 

a 



a 

o 

3 

tiO 

CO 

Oh 

n 



CO 



CO 

d) 

O 

I-H 

o 

o 

CO 

d) 

l-H 

P 

y) 



OX) 

S 

fg o 

3 ^ 

S CO 

.O 0 

&> 

W -M 

Us" 



1-ll 

2 |.| 

« ® « 0) 

S' o 

2 c o ^ 
^ S o <u 

S' « « 5 






CO 

o 

3 

o 

1-H 
4— • 

o 

g 

3 

Oh 

1^ 

CO 

1-H 

d) 



Oh 

d) 

O 



00 

o 



CO 

d) 

O 

§ 



a 

o 

4-4 

CO 



,o 

-M 

g 

■o 

d) 

4— • 

CO 

d) 

V4 

L) 

y) 

00 

g 

3 

CO 

d) 

o 

0 

CO 

4-4 

c 

d) 

1 

CO 

d) 

CO 

CO 

cO 



00 

O 

o 



o 

d) 

4-4 

a 

o 

00 

o 

[3 

3 



a 

d) 

3 

CO 

CO 

(D 

CO 

CO 

cO 

CO 

■o 

d) 

d) 



er|c 



132 



Analysis of performance Matrix 



o 

ERIC 



133 



APPENDIX E 



SUMMARY Tabulations From the 1998 STW Local Partnership Survey 



Available Upon Request 




131 



APPENDIX F 



Major issues for Future National Research 



Major Issues for Future National Research 

Although there has been considerable effort to investigate STW at the national level, a number of 
unanswered questions remain: 

• Will early partial efforts lead to more comprehensive STW approaches? 

• Do STW efforts support higher academic standards and achievement? 

• What is the potential for sustained STW systems? 

This section addresses these questions and identifies associated issues. 

Will Early Partial Efforts Lead To More Comprehensive STW Approaches? 

The three national studies of STW show considerable progress in only a few years toward 
implementing key elements of the STWOA. However, many schools are providing only some 
parts of the STW school-improvement strategy, and few students participate in a comprehensive 
career- focused high school learning experience. The National Evaluation indicates that schools 
have a long way to go to integrate comprehensive career development activities, career-related 
Academies, and paid or unpaid work experience that is linked to learning in the classroom. As 
shown in Figure 6, only about 3% of students have been involved in all three of these curriculum 
approaches. Future research must focus on how extensively schools go beyond these partial 
efforts to include all students in a multifaceted STW experience. 
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Figure 6. Student Involvement in Key STW Components in Member High 
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[National Evaluation, Figure III. 14, p. 97] 




even indicated that raising academic standards — particularly through heavy new emphases on 
standardized testing — has slowed the progress of STW-type reforms. The time is right for 
additional research to explore the impact of STW on academic standards and a broad array of 
student outcomes. 

What Is the Potential for Sustained STW Systems? 

Future prospects for LPs vary widely. As STW begins to enter a transition phase in many states, 
practitioners will need to answer questions such as the following (Progress Measures, p. 35): 

• To what extent do high levels of school and student participation enhance a partnership’s 
prospects for sustaining STW activities? 

• Is there a relationship between early success in identifying alternative funding mechanisms 
and subsequent growth and institutionalization of STW activities? 

• Do partnerships with consistent, high-quality employer participation exhibit greater success 
in sustaining STW activities? What roles and responsibilities are employers assuming to 
promote continuation of STW activities? 



Measuring and Documenting Student Outcomes in School-To-Career Presentation, Alameda County Office of 
Education, March 2000. 
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• Does sustaining the STW concept and its benefits for students require sustaining a regional 
partnership infrastructure launched with STWOA funds? Or, can the results of STW-type 
reforms endure solely through the efforts of individual schools and districts? 
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